Trials and tribulations: so many potential treatments, so few answers

Abstract

Purpose

The purpose of this review is to quantify the landscape of current clinical trials ongoing for therapies in the treatment of COVID-19. A secondary purpose is to examine the relationship between public and scientific interests in potential therapies for COVID-19.

Methods

A systematic search of clinicaltrials.gov was undertaken on April 22, 2020, to identify all currently registered clinical trials investigating potential therapies for patients with COVID-19. Public interest in the various therapies was quantified utilizing Google Trends. Public interest in hydroxychloroquine and chloroquine was plotted against the cumulative number of active clinical trials evaluating antimalarials as potential COVID-19 therapies over time.

Results

There were 341 interventional studies and 208 different therapies actively registered on clinicaltrials.gov whose primary aim is the treatment of COVID-19. The median sample size was 120 patients (range 4–6000) with 154 (45%) trials reporting a planned sample size of 100 patients or less. There was a strong positive correlation (r = 0.76, p = 0.01) between the number of registered clinical trials and the public interest in the top ten proposed therapies. Following the spike in public interest, the average number of new trials increased tenfold with respect to antimalarial therapies.

Conclusions

The relatively small sample sizes and the number of independent trials investigating similar therapies are concerning. Resources may not be being allocated based on scientific merit and may be driven by public consciousness and speculation. Moving forward, a concerted effort focused on implementing large, well-coordinated and carefully designed multi-armed clinical trials will help to ensure that the most promising therapeutic options are rigorously studied and clinically meaningful results produced.

Introduction

The current global landscape created by the COVID-19 pandemic has led to a rapid and concerted effort of researchers around the globe to determine the utility and efficacy of pharmaceutical treatments for COVID-19. This pandemic has created an explosion of clinical trials in attempts to reach any promising treatment for this virus. Potential treatments are varied; ranging from antimalarials and antivirals to natural honey. Scientists have collaborated across institution and country lines in an unprecedented fashion, to evaluate potential COVID-19 therapies [1]. Most promising among these efforts are randomized controlled trials (RCTs), which represent the highest level of evidence which informs clinical decision-making in the evidence-based medicine era.

However, with a large number of active and upcoming trials, the question should be asked if this an appropriate use of resources likely to yield any clinically meaningful data [2, 3]? The global research community is rushing to find a public health solution and potential therapies to COVID-19, but a poorly coordinated effort will likely yield poor results [2]. To date, most trials are small in scale, which limits the power of these studies and introduces biases that can confound results. Results may look promising in their current sample size but are often not generalizable to a large population scale, or vice-versa [4]. When attempting to synthesize information from small sample case series and single-centre trials, there is often heterogeneity in methodology that limits data pooling and reduces meaningful conclusions for appropriate treatment decisions [5, 6]. Therefore, fewer large-scale, methodologically sound studies are more useful than multiple, low-quality, small trials. Orthopaedic literature has shown that conducting insufficiently sized and powered trials has a risk of not only missing true effects but can produce false ones as well [7].

Previous literature has shown statistical conclusions are less likely to be legitimate with smaller samples/effect sizes, greater design flexibility, and when there are more teams independently pursuing, the same topic, particularly in emerging topics [8]. Many of these risks are applicable to the current body of COVID-19 trials. Although positive steps are being made with a more coordinated effort in the World Health Organization’s “solidarity” and the UK’s RECOVERY trial, results are likely months away [2]. Finally, with the ongoing infodemic of mainstream and social media, the question should be asked if the current body of trials is piquing public interests, or if public interests and media attention are influencing research questions typically guided by scientific merit in less pressured times [9].

The purpose of this review is to quantify the landscape of current clinical trials ongoing for therapies in the treatment of COVID-19. A secondary purpose is to examine the relationship between public and scientific interests in potential therapies for COVID-19.

Methods

A systematic search of clinicaltrials.gov was undertaken on April 22, 2020, to identify all currently registered clinical trials investigating potential therapies for patients with COVID-19. The following data was extracted: date first reported on clinicaltrials.gov, therapies investigated, and proposed sample size. Trials were classified by specific therapy and class of medication (e.g., antimalarials, antivirals). All individual trial arms were recorded to assess for the frequency of different therapies. All trials registered on clinicaltrials.gov were included, with the exception of trials that had been withdrawn, suspended, or terminated. Trials investigating therapeutic agents for prophylaxis or prevention of COVID-19 were excluded.

We also sought to quantify public interest in the various therapies utilizing Google Trends (GT). GT is a publicly available resource that tracks internet hit-search volumes. The tool determines the proportion of searches for specific search terms among all searches performed using Google. It provides a relative search volume (RSV), which is the query share relative to the total number of searches on Google, on a scale of 0 to 100, with higher numbers representing greater search volumes. The search terms included the top ten potential therapies based on number of active trials retrieved from clinicaltrials.gov. These search terms were inputted into GT using the following filters: “Worldwide,” “date,” “All Categories,” for “Web Searches.” Given that GT only allows for a comparison of 5 terms at one time, the therapy with the highest relative value (hydroxychloroquine) was identified and used as one of the five search terms in every search. The RSV for each trial was trended from Feb 5th to April 22nd and the average RSV over this time was calculated. February 5th represents the date of the first posted clinical trial on clinicaltrials.gov. The correlation between RSV and number of clinical trials for the top potential therapies was calculated via Spearman rank-order correlation coefficient.

Given the interest in antimalarials both among registered clinical trials and on GT, we sought to determine if the two were correlated. The RSV for hydroxychloroquine and chloroquine was plotted against the cumulative number of active clinical trials evaluating antimalarials as potential COVID-19 therapies over time. We calculated the average number of clinical trials added daily, both before and after peak public interest, as defined by GT.

Results

As of April 22, 2020, there were 341 interventional studies actively registered on clinicaltrials.gov whose primary aim is the treatment of COVID-19. The first registered trial was posted on February 5, 2020. There are 208 different therapies under investigation, including single and combination treatments (Fig. 1). The most common therapies being studied are hydroxychloroquine (n = 52), plasma (n = 25), and hydroxychloroquine + azithromycin (n = 19). The most popular type of therapy being investigated is antimalarials, with 107 different study arms either alone or in combination therapy. The median sample size was 120 patients (range 4–6000) with 154 (45%) trials reporting a planned sample size of 100 patients or less.

Fig. 1.

Potential therapeutic options. The size of each area of the sunburst chart corresponds with the proportion of trials in which the treatment is being tested

Hydroxychloroquine + azithromycin, hydroxychloroquine, and plasma were the most popular Google search terms out of the top ten potential therapies. There was a strong positive correlation (r = 0.76, p = 0.01) between the number of registered clinical trials and the public interest in the top ten proposed therapies.

Regarding antimalarials, worldwide public interest spiked on March 19, 2020. There was an average of 0.2 new antimalarial trials registered per day prior to the spike in public interest (February 7–March 18, 2020). Following the spike in public interest, the average number of new trials increased tenfold to 3.3 trials registered/day (March 20th–April 20th). Figure 2 demonstrates the relationship between worldwide public interest and clinical trial registration for antimalarials.

Fig. 2.

Relationship between public interest and registered clinical trials investigating antimalarials over time

Discussion

This present review identified 341 interventional trials in the treatment of COVID-19 registered on clinicaltrials.gov in a short 11-week span. Key findings identified included (1) large number of different therapies being investigate, (2) large number of duplicate or similar trials being conducted, particularly on antimalarials, (3) the majority of trials have small proposed sample sizes, and (4) a substantial increase in trial registration after public interest spiked regarding antimalarial therapy.

Scientists, institutions, and governing bodies should be lauded for mobilizing research teams and implementing clinical trials on a scale that has never been seen before. However, the vast number of different therapies being trialed alongside the higher number of similar but distinct trials underway must be a pause for thought—are hundreds of relatively small trials beneficial or necessary [2, 3]?

Experts have suggested that the unprecedented number of clinical trials underway may not be the best utilization of resources [2, 3]. The large number of rapidly produced, relatively small, duplicate trials poses several potential problems. There is a tendency in emerging “hot” topics, that research findings are less likely to be true. In emerging topics, such as the current pandemic, there is a tendency for competing research teams to prioritize the dissemination of impressive positive results. Only after the publication of positive findings, are negative results pursued and published [10–12]. When multiple similar, but distinct, trials with uncoordinated study protocols are studying the same hypothesis, it increases the probability of false-positives [13].

The relatively small median sample size and number of trials with sample sizes under 100 are certainly concerning. Under-powered trials have the potential to over or underestimate the effective size and steer clinicians away from the most promising treatments [7, 14, 15]. There are numerous examples in the literature of promising therapies later disproven by larger trials [4, 12]. An early review of COVID-19 trials demonstrated an unacceptable quality of the sample size calculations and urged researchers to ensure statistically appropriate sample sizes to reduce the risk of under-powered studies [16]. Small, heterogenous trials also make it more challenging to pool results and make findings in meta-analyses less reliable and more difficult to interpret [17, 18]. Prioritizing larger, multi-armed trials within a common statistical framework reduces risk of bias and allows for the cross-comparisons [19].

The public interest and number of clinical trials involving antimalarials have far outpaced other potential therapies. The rise of clinical trials following the spike of public interest begs the question—are priorities and resources being allocated based on scientific merit, or driven by public consciousness and speculation? Public interest in hydroxychloroquine spiked after a non-randomized open-label clinical trial in France demonstrated reduced viral loads in COVID-19 patients who received hydroxychloroquine and azithromycin [20]. These findings were widely promoted by prominent political leaders which spiked both public and scientific interest. However, the study was deeply flawed methodologically, and more recent studies have demonstrated no benefit [21]. Currently, no governing bodies recommend hydroxychloroquine as a potential therapeutic agent, and it is not being recommended for use outside clinical trials [22]. The emphasis on clinical trials investigating hydroxychloroquine may reduce the ability to test other promising therapies through a reduction in available resources and recruitment competition.

This review is strengthened in its comprehensive inclusion of all registered therapeutic trials, allowing for an understanding of the current scientific landscape. It is limited by the surface-level nature in which the data was collected and grouped as well as the utilization of a single database in the interest of efficiency.

Moving forward, a concerted effort focused on implementing large, well-coordinated and carefully designed multi-armed clinical trials will help to ensure that the most promising therapeutic options are rigorously studied and clinically meaningful results produced.

Author contributions

Author (A.G.) contributed to (1) the conception and design of the study, or acquisition of data, or analysis and interpretation of data, (2) drafting the article or revising it critically for important intellectual content. Authors (N.N. and M.Z.) contributed to the conception and design of the study, or acquisition of data, or analysis and interpretation of data. Authors (S.E., C.G., A.P., M.B.) contributed to (1) the conception and design of the study, or acquisition of data, or analysis and interpretation of data, (2) drafting the article or revising it critically for important intellectual content.

Data availability

Data can be made available upon request.

Compliance with ethical standards

Conflict of interest

Author (M.B.) reports personal fees from AgNovos Healthcare, personal fees and other from Sanofi Aventis, personal fees and other from Smith & Nephew, personal fees from Stryker, grants from DJ Orthopedics, and other from Ferring Pharmaceuticals outside the submitted work. No other authors report conflicts of interest.