Research response to coronavirus disease 2019 needed better coordination and collaboration: a living mapping of registered trials

Abstract

Objectives

Researchers worldwide are actively engaging in research activities to search for preventive and therapeutic interventions against coronavirus disease 2019 (COVID-19). Our aim was to describe the planning of randomized controlled trials (RCTs) in terms of timing related to the course of the COVID-19 epidemic and research question evaluated.

Study Design and Setting

We performed a living mapping of RCTs registered in the WHO International Clinical Trials Registry Platform. We systematically search the platform every week for all RCTs evaluating preventive interventions and treatments for COVID-19 and created a publicly available interactive mapping tool at https://covid-nma.com to visualize all trials registered.

Results

By August 12, 2020, 1,568 trials for COVID-19 were registered worldwide. Overall, the median ([Q1–Q3]; range) delay between the first case recorded in each country and the first RCT registered was 47 days ([33–67]; 15–163). For the 9 countries with the highest number of trials registered, most trials were registered after the peak of the epidemic (from 100% trials in Italy to 38% in the United States). Most trials evaluated treatments (1,333 trials; 85%); only 223 (14%) evaluated preventive strategies and 12 postacute period intervention. A total of 254 trials were planned to assess different regimens of hydroxychloroquine with an expected sample size of 110,883 patients.

Conclusion

This living mapping analysis showed that COVID-19 trials have relatively small sample size with certain redundancy in research questions. Most trials were registered when the first peak of the pandemic has passed.

What is new?

Key findings

• •Most of clinical trials of COVID-19 are planned after the peak of the epidemic.
• •These trials are mainly single-centered, open-labeled, and have relatively small sample size.
• •There is a notable redundancy in research questions.

What this study adds?

• •We have created a living mapping that visualizes all clinical trials of COVID-19.
• •The living mapping supports researchers and decision makers in identifying research gaps, thus planning research of high priority.

What is the implication and what should change now?

• •Research community needs a better coordination in research planning to ensure that all potential treatments for COVID-19 are evaluated with robust methodology.
• •The living mapping provides a tool to monitor status of research and enhance research collaboration and interaction in medical and scientific community to avoid research waste.

1. Introduction

In December 2019, an outbreak of pneumonia caused by a novel coronavirus started in Wuhan, Hubei Province in China. The disease was later determined to be SARS-CoV-2 infection or coronavirus disease 2019 (COVID-19) [1]. In early March 2020, the disease had spread to more than 100 countries and territories [2]. On March 12, 2020, the World Health Organization (WHO) declared the outbreak a pandemic [3]. To respond to this emergency, researchers all over the world began to actively engage in research activities to develop and evaluate preventive and therapeutic agents for COVID-19.

Given this unprecedented context, we aimed to inform decision makers and researchers in near real-time about current research efforts, research gaps, and overlap. A mapping of all research efforts is imperative to support researchers and decision makers to monitor status of research response to the epidemic and integrate emerging evidence in research planning timely to ensure that all potential treatments are evaluated, while avoiding waste in resources invested. For this purpose, we performed a living mapping of all registered randomized controlled trials (RCTs) investigating interventions to prevent and treat COVID-19. This living mapping is updated every week, and the results are publicly available at https://covid-nma.com/.

This article describes the planning of RCTs in terms of timing related to the course of the pandemic and research questions.

2. Methods

This mapping is part of the COVID-NMA project, which also includes living systematic reviews and living network meta-analyses of studies of COVID-19 [4,5]. The protocol of this project is available at https://zenodo.org/record/3903347#.XwLasUBuI2x.

2.1. Data sources

Our data are obtained from the WHO International Clinical Trials Registry Platform (ICTRP) (https://www.who.int/ictrp/en/), an international registry that assembles information on clinical trials registered in 17 primary registries [6]. The WHO ICTRP has created a database dedicated to all clinical trials evaluating interventions to prevent and treat COVID-19. The database is updated weekly and is publicly available.

2.2. Eligibility criteria

Whenever the database is updated, we use Hypertext Preprocessor (PHP) programming language to identify studies that are newly registered in the database. Two researchers (VN and GF) systematically search the platform every week to identify new eligible RCTs for data extraction. All RCTs assessing the efficacy and safety of interventions for preventing or treating COVID-19 and patients in the postacute period are included.

We exclude observational studies, case series, and nonrandomized or single arm studies (i.e., diagnostic test studies). We also exclude studies (1) evaluating interventions to reduce psychological distress caused by the COVID-19 outbreak or (2) assessing herbs, homeopathy therapy, and traditional Chinese medicine (TCM) (with only TCM in two groups or TCM plus standard of care).

2.3. Data extraction

A standardized data collection form is used to collect data describing the RCTs. Several data items are available from the WHO ICTRP database, such as registration number, countries where trials are conducted, recruitment status, inclusion and exclusion criteria, primary outcomes, and sample size. A team of 11 trained data collectors independently retrieve other information from the trial registration such as study aim, number of arms, type of participants, and information related to experimental treatments and comparators (i.e., treatment name and treatment type). Two researchers (VN and GF) verify the quality of the data and ensure the consistency of data entered in the database.

We classify study aims as evaluation of prevention interventions, COVID-19 treatments, and postacute period interventions. In RCTs evaluating preventive interventions, participants are classified as healthy volunteers, health workers, and high-risk patients. Patients in RCTs assessing COVID-19 treatments are classified by disease severity (i.e., mild, moderate, severe, and critical). Clinical criteria for classifying disease severity are provided in Appendix 1. The full list of treatment types is provided in Appendix 2.

2.4. Monitoring the recruitment status

When the database of the WHO ICTRP is updated every week, we use PHP programming language to identify RCTs with changes in recruitment status (e.g., from not recruiting to recruiting) and update our database accordingly.

2.5. Data on the course of the epidemic

The COVID-19 database maintained by Our World in Data (https://ourworldindata.org/coronavirus-source-data) was used to visualize the evolution of the pandemic over time. The database is updated daily and includes the number of confirmed cases, deaths, and testing data. We considered only data related to the number of confirmed cases and deaths.

2.6. Data analysis

We created an online interactive mapping tool to visualize the data of trials registered. The interactive mapping was developed with D3.js [7] as an Observable notebook [8]. The projection for the map used was implemented in JavaScript [9]. We also used time series plotting to visualize the evolution of COVID-19 research over time. This visualization was performed in R v3.4.2 (the R Foundation Statistical Computing, Vienna, Austria).

3. Results

3.1. General characteristics of the registered COVID-19 RCTs

Up to August 12, 2020, there were 4,956 studies registered on the WHO ICTRP; 1,568 were RCTs, of which 878 (56%) were recruiting. Overall, 35 trials were completed; only 4 trials gave access to the results. In our database, 735 (47%) trials are single-center. Trials are being conducted in Asia (32%), Europe (28%), and North America (21%). Most trials use a parallel study design (91%) and 705 (45%) an open-label design (Table 1 ). Most trials (n = 1,333; 85%) focus on interventions for treating COVID-19. Only 223 (14%) consider prevention, and only 12 (0.8%) evaluate postacute period care. The median ([Q1–Q3]; range) sample size is 540 ([200–1,600]; 30–130,000) for trials evaluating prevention and 100 ([60–269]; 10–12,000) for those evaluating treatment. Overall, 52 trials are conducted in multiple regions.

Table 1.

Characteristics of registered COVID-19 trials in the WHO International Clinical Trials Registry Platform at the time of analysis (n = 1,568)

Study characteristics
Region
Asia	498 (32)
Europe	436 (28)
North America	324 (21)
Latin America	114 (7)
Africa	73 (5)
Oceania	23 (2)
Multiple regions	52 (3)
Not reported	48 (3)
Recruitment status
Not recruiting	643 (41)
Recruiting	878 (56)
Completed	35 (2)
Suspended	6 (0.4)
Terminated	5 (0.3)
Withdrawn	1 (0.1%)
With results available	32
Number of centers
Single center	735 (47)
Multiple centers	638 (41)
Not reported	195 (12)
Study design
Parallel	1,426 (91)
Adaptive	53 (3)
Sequential	42 (3)
Factorial	29 (1.8)
Crossover	15 (1)
Cluster	3 (0.2)
Masking
Open label	706 (45)
Blinded label	798 (51)
Not reported	64 (4)
Study aim
Prevention	223(14)
Treatment	1,333 (85)
Postacute period care	12 (1)
Sample size, median
Trials evaluating preventive interventions	540 (200–1,600)
Trials evaluating treatment	100 (60–269)
Trials evaluating postacute period interventions	100 (60–121)

Data are n (%) or median (Q1–Q3).

The interactive map (https://covid-nma.com/dataviz/#) allows users to interact with data by selecting different parameters such as locations, severity of patients included in trials, and type of treatment evaluated to visualize all studies of interest.

In certain countries, the sample size is relatively small for trials evaluating COVID-19 treatments (Table 2 ). In the United States, the country with the highest number of trials registered, 113 (47%) trials have less than 50 patients per trial arm and 86 trials (36%) have more than 100 patients per trial arm. In China, 86 trials (54%) have less than 50 patients per arm and 35 trials (22%) with more than 100 patients per arm. In Europe, Spain registered the highest number of trials with 39 trials (42%) having less than 50 patients per arm and 25 trials (27%) having more than 100 patients per arm. By contrast, in the United Kingdom, 52% (22/42) of trials have more than 100 patients per arm.

Table 2.

Sample size of registered COVID-19 trials in countries with the highest number of COVID-19 cases at the time of analysis

Country	Number of trials registered	Median sample size	Number of trials with more than 100 patients per arm
United States	238	103 (50–300) [10–10,000]	86 (36)
China	160	90 (58–160) [12–520]	35 (22)
Iran	140	60 (40–100) [10–3,000]	11 (8)
Spain	93	104 (60–200) [18–3,040]	25 (27)
France	78	189 (100–428) [20–3,140]	36 (46)
United Kingdom	42	275 (64–471) [20–12,000]	22 (52)
Italy	30	162 (100–376) [50–2,712]	10 (33)
India	47	100 (45–183) [20–1,500]	10 (21)
Brazil	44	196 (84–446) [30–1,968]	21 (48)

Data are n (%) and median (Q1–Q3) [range].

3.2. Timing of research response to the evolution of the pandemic

Fig. 1 represents a strip plot of all registered RCTs over time in terms of the first confirmed case for each country. The interactive version of this figure (https://covid-nma.com/research_delay/) allows for representing and aligning trial registration by the first confirmed case, the 100th confirmed case, the first death, and the 10th death. Overall, the median ([Q1–Q3]; range) delay between the first case recorded in each country and the first RCT registered was 47 days ([33–67]; 15–163). Similarly, the median ([Q1–Q3]; range) delay between the first death and first trial registered was 21 days ([11–38]; −7 to 108).

Fig. 1.

Registered trials of COVID-19 over time in countries with more than 50,000 confirmed COVID-19 cases.

Fig. 2 shows the number of registered trials evaluating COVID-19 treatment overtime, the cumulative number of patients to be recruited and the number of new COVID-19 cases per day for the 9 countries with the highest number of trials registered at the time of analysis. This figure does not consider multinational trials because the expected number of patients to be recruited in each country is not available.

Fig. 2.

Expected sample size in registered trials evaluating COVID-19 treatments in comparison with the evolution of the epidemic in each country (only single-country trials are presented).

Overall, the cumulative expected sample size for all trials evaluating COVID-19 treatment was 378,649 patients. In the United States, 90/238 (38%) trials were registered before the first epidemic peak (i.e., April 26, 2020), representing 44% of the total number of patients to be recruited in all trials. In China, 41/160 (26%) trials were registered before the peak (i.e., February 13, 2020), representing 33% of the total number of patients to be recruited in all trials. In Europe, Spain registered only 2/93 trials (2%) before the peak (i.e., March 27, 2020). In France, the first trial was registered only 8 days before the peak (i.e., April 01, 2020). Eight trials (10%) registered before the peak in France accounted for 28% of the total number of patients to be recruited in all trials. In the United Kingdom, 6/42 (14%) trials were registered before the peak (i.e., April 12, 2020), representing 40% of the total number of patients to be recruited in all trials. In Italy, no trial was registered before the peak on March 22, 2020.

3.3. Research questions and interventions evaluated

3.3.1. Trials evaluating preventive interventions

In our database, among 223 trials evaluating preventive interventions, the most common chemoprophylaxis evaluated is antimalaria drugs (68 [36%] trials; 62 assessing hydroxychloroquine as monotherapy expecting to recruit 93,267 participants and 6 assessing chloroquine as monotherapy expecting to recruit 136,770 participants). In total, 89 trials are evaluating different types of vaccines; 27 evaluate Bacille Calmette-Guérin vaccine to prevent COVID-19, and 52 trials evaluate vaccines specifically developed for coronavirus.

3.3.2. Trials evaluating COVID-19 treatments

Overall, 1,333 trials evaluating COVID-19 treatments are evaluating antimalaria drugs (254 trials, 19%), different regimes of antivirals (236 trials, 18%), and monoclonal antibodies (133 trials, 10%). Fig. 3 shows the evolution overtime of the 8 most evaluated therapeutic agents. In the interactive mapping version, users are able to select the number of therapeutic agents to be shown, countries, and number of trials evaluating each treatment or expected patients to be recruited. After March 22, 2020, the number of trials evaluating hydroxychloroquine increased greatly. Up to August 12, 2020, 142 trials worldwide were planned to assess hydroxychloroquine, with an expected sample size of 34,080. Lopinavir/ritonavir is the second most common agent evaluated, with 48 trials and an expected sample size of more than 12,734, followed by tocilizumab, with 41 trials and an expected sample size of 6,121. In total, 28 trials are planned to evaluate remdesivir with 7,365 patients.

Fig. 3.

Cumulative number of registered COVID-19 trials and patients planned to be recruited over time for the 10 most commonly assessed therapeutic agents.

Fig. 4 presents the 8 most evaluated therapeutic agents in 6 countries. In our database, hydroxychloroquine is the most commonly tested agent across these countries, and trials evaluating different regimes of hydroxychloroquine expect to involve the highest number of patients. In Brazil, 15/44 trials are evaluating different regimes of hydroxychloroquine and expect to recruit 4,121 patients, representing 28% of all expected patients to be recruited in Brazil at the time of analysis (14,805 patients). In France, 6,003 patients are expected to be recruited in trials of hydroxychloroquine, representing 20% of all expected number of patients to be recruited (30,079 patients). In the United States, 9,325 patients are expected to be recruited in trials of hydroxychloroquine, representing 14% of all expected number of patients to be recruited (70,596 patients).

Fig. 4.

Cumulative number of expected patients in registered COVID-19 trials evaluating therapeutic agents in 6 countries with the highest number of COVID-19 trials registered.

Fig. 5 represents a mapping of trials evaluating the five most frequently tested treatments in regard to severity of patients recruited in trials and types of comparators.

Fig. 5.

Mapping of the five most frequently evaluated treatments.

4. Discussion

4.1. Summary of findings

Our living mapping of RCTs of prevention and treatment of COVID-19 shows a substantial waste of research because of lack of coordination and collaboration in the research response to the pandemic. While WHO and European Medicines Agency called for efforts to prioritize large, multicenter, and multiarm trials to provide meaningful and interpretable evidence, most trials registered had only a single center involved and a relatively small sample size [10]. Although research community have had responded to the epidemic at an unprecedented rate to set up trials rapidly after the first confirmed case, our mapping shows that many trials across different countries were planned when the first peak of the epidemic had passed. In Europe, most of the patients to be recruited are expected to come from trials registered after the first peak of the epidemic. This situation is particularly problematic because the time frame for including patients is quite short, only a few weeks during the epidemic. Trials registered when the epidemic has wound down might not succeed in recruiting a large enough sample for a clear conclusion. At the time of this analysis, no trial in France, Spain, and Italy had released official results, which might reflect the difficulties in recruitment due to the late recruitment during the first peak. Furthermore, trial results will probably be published after the epidemic has passed, and countries where the trials were conducted might not have the direct benefits to improve clinical practice at the time of the epidemic [[11], [12], [13], [14], [15]].

The planning of trials in response to COVID-19 has a notable redundancy in research questions. In March 2020, hydroxychloroquine received tremendous attention after the results of an observational study in France were published that generated a huge debate [16]. After this publication, the US president highlighted this treatment as being a “game changer,” despite the lack of consistent data from RCTs [17]. On March 28, 2020, the US Food and Drug Administration approved hydroxychloroquine for treating COVID-19 [18]. Just after this, the number of trials evaluating different regimens of hydroxychloroquine increased markedly, with the number of patients expected to be recruited representing 10% of the total number of patients to be recruited in all trials registered. This massive concentration of research efforts for one therapeutic agent not only competes with research examining other hypotheses for scarce resources but also impairs recruitment in trials evaluating other potential treatments [19]. The second most frequently evaluated treatment was lopinavir/ritonavir, with 39 trials representing 4% of the patients to be included in trials of COVID- 19 treatment. Nevertheless, a large RCT, RECOVERY, conducted in the United Kingdom, showed no beneficial effect of hydroxychloroquine or lopinavir/ritonavir [11]. The WHO recently decided to discontinue the hydroxychloroquine and lopinavir/ritonavir arms of the Solidarity trial after the interim results were communicated [20]. Indeed, clinical trials need to be replicated in different settings to have confidence in the results. However, the number of trials planned was disproportionate to the needs. Consequently, resources invested in trials evaluating hydroxychloroquine and lopinavir/ritonavir would likely be wasted.

4.2. Implications

For many emerging infectious diseases such as COVID-19, the time when the epidemic occurs is the only opportunity to conduct research and generate evidence about the efficacy of therapeutic treatments and preventive measurements. This mapping shows that many COVID-19 trials might have missed the first peak of the pandemic. This issue reflects the challenges that researchers might encountered in planning and conducting trials under a complex context of the pandemic. At the early stage of the pandemic, there was limited data on potential treatments to be evaluated in a trial, no core outcome set available for COVID-19 to guide the selection of outcome. Logistical challenges in producing placebo might also influence the choice of trial design. Further, finding trial personnel is not an easy task as clinicians and nurses are overburdened with patient care. Although multicentered trials could help to increase recruitment rates, obtaining funding, regulatory, and ethical approvals from multiple sites could be challenging in the context of the pandemic, which delayed the process. However, the lesson from previous epidemics such H1N1 and Ebola highlighted the importance of starting and completing trials during the peak of the epidemic to ensure successful recruitment and provide evidence timely to patient care. To overcome logistic and methodological challenges, the two large trials RECOVERY and Solidarity used relatively simple protocol with a straightforward outcome of all-cause mortality to reduce burden of data collection. The RECOVERY trial successfully used a robust adaptive design to test a range of different treatment options with minimized administrative tasks to avoid burdening the health care system. Collaboration between researcher centers to boost recruitment plays a crucial role in addressing the fast evolution of the epidemic. The RECOVERY trial with a network of 175 hospitals in the United Kingdom rapidly enrolled thousands of patients to provide evidence on effectiveness of hydroxychloroquine and dexamethasone to the medical community [11,21]. With the scarcity of research resources, we must coordinate research efforts, identify gaps that need further research and ensure that all promising treatments are being evaluated [22]. National regulatory bodies, ethical review boards and funders should prioritize and facilitate the conduct of large, multicenter, and multiarm trials. Regulatory and health authority should provide timely guidance to clinicians to avoid off-label drug uses based on anecdotal evidence which might cause difficulties to trial planning and recruitment [23,24].

This study highlights the importance of clinical trial registries, an underused resource, to monitor the state of research for improving the organization of research efforts [[25], [26], [27]]. Our interactive living mapping of COVID-19 research was designed to help decision makers use data from clinical registries for an up-to-date picture of all research questions being investigated so as to prioritize research and avoid waste in research [28]. Furthermore, this interactive mapping tool might also enhance collaboration in research to reduce redundancy and competition in trial organization [19,29].

4.3. Limitations

In this analysis, we visualized trial registration over time by using the registration date rather than the actual starting date of recruitment because the ICTRP database did not distinguish the actual starting date from the expected starting date for each trial. In addition, investigators might not regularly update the status of recruitment on trial registries. For example, the trial ChiCTR2000029544 was reported as “Not recruiting” on the registry, but the results of the trial were published [30]. Furthermore, the structure of reporting is heterogeneous across the primary registries, which affects the quality of reporting [31]. Investigators might register one trial in more than one registry under different titles or investigator names, such duplicates are almost systematically detected by the ICTRP while a very few may remain undetected. Finally, we did not assess the risk of bias for each trial registered as information in trial registration is inadequate to enable a comprehensive assessment.

5. Conclusions

We have created a living mapping tool to keep track of the evolution of research on COVID-19 for supporting decision makers in prioritizing and planning research. This mapping analysis showed that many COVID-19 trials were registered after the first peak had passed and a need to improve the organization of research efforts to avoid research redundancy. Visualizing ongoing research can enhance the collaboration and interaction between research communities that can go beyond the COVID-19 crisis.