Abstract
The coronavirus disease 2019 (COVID‐19) pandemic has led to a dramatic impact worldwide and presented unprecedented challenges for clinical and translational medicine. We assess the impact of COVID‐19 on submitted and completed interventional clinical trials that have been registered on ClinicalTrials.gov. After classifying over 85% of the registered clinical trials by their source, we carefully model the number of submitted and completed trials before and after March 2020. Overall, we find minimal impact of COVID‐19 on the number of submitted clinical trials, although a much more substantial impact is observed for completed clinical trials. We also show that clinical trials with a pharmaceutical sponsor were more successful at completing trials during the pandemic compared to the trials with academic/hospital/government sponsors.
Study Highlights.
WHAT IS THE CURRENT KNOWLEDGE ON THE TOPIC?
There have been no published papers to date to assess the effect of coronavirus disease 2019 (COVID‐19) on the number of completed and submitted interventional trials. Medidata reports have shown that comparing the average number of new patients entering trials per study‐site trials in 2020 to 11 months of 2019 showed a decrease from −59% in April to −10% in July and then back to −20% in August. Our results are consistent with Medidata findings as the decrease in enrollments could be associated with a decrease in number of submitted and completed clinical trials.
WHAT QUESTION DID THIS STUDY ADDRESS?
This study addresses the following key question: “How has COVID‐19 affected the number of submitted and completed interventional clinical trials?”
WHAT DOES THIS STUDY ADD TO OUR KNOWLEDGE?
This study analyzed and visually illustrated the association of COVID‐19 with the number of submitted and completed interventional clinical trials in different regions across pharmaceutical and academic sectors.
HOW MIGHT THIS CHANGE CLINICAL PHARMACOLOGY OR TRANSLATIONAL SCIENCE?
This study shows that COVID‐19 has substantially impacted translational science by hampering the completion of clinical trials. However, with governmental support, as seen in Egypt, the detrimental effects of COVID‐19 on clinical trial research can be avoided.
INTRODUCTION
The coronavirus disease 2019 (COVID‐19) pandemic has dramatically altered our lives having already taken the lives of over 1.4 million people and infecting a substantial fraction of the world’s population. This pandemic has also caused a hefty economic burden; just in the United States alone, 60 million unemployment claims have been filed since March 2019 with estimated total economic losses surpassing $16 trillion. 1 The COVID‐19 pandemic has also caused unmatched disruption to research and clinical trials worldwide and has hindered the ability of conducting these trials effectively and safely. Social distancing and lockdowns that aimed to reduce the spread of the virus have also challenged the recruitment and conduction of clinical trials.
Early reports by Medidata have showed that over 1000 clinical sites surveyed indicated a negative impact of COVID‐19, with two‐thirds of respondents having stopped or will soon stop patient recruitment for ongoing trials, a third have halted randomization, and about half are now delaying or will delay their studies. 2 A recent paper analyzed ClinicalTrials.gov data and surveys from 245 clinical trial investigators, confirming a decrease in new trial starts from January to May 2020 with slow recovery in June and July in addition to an 80% decrease in new patients entering trials per site in April 2020 compared with April 2019. 3 They also report that the number of trials suspended due to COVID continued to peak and stabilized between February and March at around 1200. Consequently, many researchers were pulled away from working on clinical trials to work in emergency medical care, especially during the first months of the pandemic, and many ongoing trials shifted and made alternative plans in conjunction with funders and institutions. 4 More than 82% of suspended clinical trials between March 1 and April 26 2020 reported the reason for the suspension as the COVID‐19 pandemic. 5 Oncology trials in particular were highly affected, with 109 of 389 disrupted clinical trials being in oncology and less than 20% of the institutions in the United States and Europe continued active normal trial enrollment for oncology trials compared with 60% in Asia. 6 The Mayo Clinic, which is one of the largest clinical trial centers in the United States, is under extreme financial strain due to the current COVID crisis, and this has impacted the stem cell trials’ budgets as well as temporary furloughs and reduction in full‐time equivalent (FTE) levels for staff; in particular, 20 regenerative medicine clinical trials were halted. 7 The situation could be much worse for clinical trial centers with fewer resources and more trials could be postponed.
This paper aims to understand the impact of COVID‐19 on the number of submitted and completed interventional clinical trials by analyzing 117,000 clinical trials from ClinicalTrials.gov. In particular, we model the number of clinical trials submitted and completed across different regions and sectors, while controlling for seasonality and other potentially confounding variables.
METHODS
General approach
For newly submitted/completed trials on ClinicalTrials.gov, we associate the trial with a region (United States/Europe/Asia/Other) based on the primary country of the source listed for the clinical trial. We also identify each source as being a pharmaceutical/biotech/for‐profit company versus an academic medical center/hospital/non‐for‐profit foundation; we simply refer to this identification as pharmaceutical versus academic sector. We ultimately classified over 85% of the 137,000 interventional clinical trials from March 2016 through October 2020.
Our approach considers two separate outcomes: the number of submitted and the number of completed interventional trials reported each month to ClinicalTrials.gov. For completed trials, we only consider “actual” completed trials, not “anticipated” completed trials. Each trial has two completion dates provided—an overall study completion date and a primary outcome completion date. Both outcomes were analyzed to assess the sensitivity between the two outcomes. The Clinical Trials Transformation Initiative (CTTI) maintains aggregate content of ClinicalTrials.gov, and they provide monthly copies of the database on their website. We compare data from the December 1, 2020, database to the December 1, 2019, database to minimize the confounding effects of time.
Starting with the December 1, 2020, clinicaltrials.gov database, we compare 36 months of data prior to March 2020—the “pre‐COVID era”—to 7 months of data after March 2020 (April through October)—the “post‐COVID era.” We minimize the skewness in the data by log‐transforming the outcome measure. It is likely that the time of year at which clinical trials are submitted/completed follow certain seasonal patterns (due to less busy seasons, availability of funding, etc.), so our model carefully removes seasonal patterns from the data. More specifically, seasonal effects were adjusted by using pre‐COVID era data and extending the estimated seasonal effects to the post‐COVID data. We repeat these same steps with the December 1, 2019, clinicaltrials.gov database by comparing 36 months of data prior to March 2019 to 7 months of data after March 2019. At the end of this process, we end up with seasonally adjusted monthly data from the 2020 data extract and an analogous set of data from the 2019 data extract to serve as a reference.
For a given region (United States/Asia/Europe) and sector (pharmaceutical/academic) we filter the 2020 and 2019 datasets accordingly. Then, for the month (, we calculate the seasonally adjusted outcome measure (either number of submitted or number of completed clinical trials) from the 2020 data subset and also calculate the analogous measure from the 2019 data subset. This gives rise to a data structure with three selections for region (United States/Europe/Asia), two selections for sector (academic/pharmaceutical), and 43 monthly values. The two final datasets (one for submitted trials and one for completed trials) analyzed consist of 258 rows and 6 variables. These datasets are provided as [Link], [Link], [Link].
Using these carefully prepared datasets, we apply a multiple linear regression model to assess the effects of COVID‐19, region, and sector on submitted and completed clinical trials. All analyses and graphics were generated using R version 4.0.2, and all source code to reproduce the tables, graphics, and analyses are available on GitHub at https://github.com/arthurberg/COVID_Clinical_Trials.
Technical approach
The entire clinical trial database from ClinicalTrials.gov is uploaded monthly by the CTTI and made readily available for download on their website at https://aact.ctti‐clinicaltrials.org/download. This analysis uses two of the monthly extracts: one from December 1, 2020, (“20201201_pipe‐delimited‐export.zip”) and one from December 1, 2019, (“20191201_pipe‐delimited‐export.zip”). For this analysis, we just utilize the “studies.txt” tables, which contains the variables study ID, date of submission, date of completion, study type (interventional/observational etc.), measure (submitted or/and completed), and source.
We filter the data to only include interventional trials and trials that were submitted or completed after March 2016, excluding the month of March 2020. We then associate each source with a country and sector (pharmaceutical/academic) based on the listed source of the trial. The specific mappings of the 3274 sources are provided in the [Link], [Link], [Link].
Seasonality of the log‐transformed pre‐COVID monthly outcome data was estimated with a fourth‐order Fourier series model using the function fourier in the R package forecast. 8 , 9 Figure 1 demonstrates this process for submitted clinical trials in which the primary location of the source is the United States. The March 2020 datapoint is excluded and the post‐COVID data values (depicted in red) are seasonally corrected using the pre‐COVID Fourier fit (depicted in magenta).
FIGURE 1.
This figure depicts the method of using a Fourier series fit of pre‐coronavirus disease (COVID) data to correct for seasonality. The data depicted are log monthly clinical trials whose source is primarily in the United States
After seasonally adjusting the log monthly counts, we attempt to correct for the confounding variable of time in the data by using the December 1, 2019, dataset—a dataset that was prepared by CTTI exactly 1 year prior. We follow the exact same seasonal correction approach, including removing the March 2019 data point and treating the April 2019 through October 2019 data points as though they were post‐COVID. Figure 2 takes seasonally adjusted submitted trials and compares the 2020 data to the 2019 data. This demonstrates the confounding effects of time on the dataset—a drop off is observed in both the 2020 and 2019 datasets. To provide a visualization of the effects of COVID‐19 that controls for the confounding factor of time, we graph the difference between the 2020 and 2019 data points and use a linear regression model with different slopes before and after March 2020 (COVID‐19 outbreak) but with a common intercept at March 20, 2020. This visual depiction of the data is equivalent to an interrupted time series analysis. 10 Confidence intervals are calculated and graphed using predict.lm and spline functions in R.
FIGURE 2.
This figure depicts the method of controlling for the confounding effects of time on seasonally adjusted data
The adjusted outcome, which we refer to as below, is the difference in seasonally adjusted log counts between the 2020 and 2019 datasets. Statistical significance of the impact of COVID‐19 is assessed with analysis of variance comparing model 1 with model 2:
where represents an indicator function specifying if the given month () is pre‐COVID or post‐COVID.
We also model the seasonally adjusted log counts in a linear regression model. The response variable, which we refer to as below, is seasonally adjusted log monthly counts of submitted/completed interventional trials. Predictor variables include region, sector, time, and a pre/post COVID‐19 indicator. The model also controls for the confounding effects of time by including the analogous 2019 outcome data, which we refer to as below, as a baseline covariate. The full analytical model is presented below.
RESULTS
Table 1 provides an overview of the number of submitted and completed clinical trials from April through October in 2019 and 2020. The 2019 counts are pulled from a December 1, 2019, data extract of ClinicalTrials.gov, whereas the 2020 counts are pulled from a December 1, 2020, data extract of ClinicalTrials.gov. To better assess the changes in the number of submitted and completed trials, the 2020 dataset is considered with and without COVID‐19 related clinical trials excluded. The results in Table 1 show that although there is a slight increase in the overall number of clinical trial submissions between April and October of 2020 compared to 2019, but with COVID‐19 related trials removed, there is a 10% decrease in the overall number of submitted trials. COVID‐19 has had a somewhat stronger impact on the number of completed clinical trials in 2020 with Table 1 showing a decrease ranging from 13% to 23% depending on the completion type or whether or not COVID‐19 trials are excluded. We see that the consequence of COVID‐19 on clinical trials is twofold: first, it causes a disruption of ongoing trials and makes it more challenging for new trials to be submitted and implemented, and second, it has led to a substantial number of new trial submissions that involve COVID‐19 outcomes.
TABLE 1.
Counts of the number submitted and completed clinical trials in 2019 and 2020
| 2019 Data |
2020 Data COVID‐19 trials included |
2020 Data COVID‐19 trials excluded |
|||||||
|---|---|---|---|---|---|---|---|---|---|
| Hosp | Pharma | Total | Hosp | Pharma | Total | Hosp | Pharma | Total | |
| Submitted | |||||||||
| Overall | 9563 | 1784 | 11,347 | 9807 (2.6%) | 1796 (0.7%) | 11,603 (2.3%) | 8640 (−9.7%) | 1607 (−9.9%) | 10,247 (−9.7%) |
| United States | 3413 | 786 | 4199 | 3335 (−2.3%) | 721 (−8.3%) | 4056 (−3.4%) | 2961 (−13.2%) | 637 (−19%) | 3598 (−14.3%) |
| Europe | 2683 | 519 | 3202 | 2548 (−5%) | 515 (−0.8%) | 3063 (−4.3%) | 2113 (−21.2%) | 461 (−11.2%) | 2574 (−19.6%) |
| Asia | 2259 | 450 | 2709 | 2497 (10.5%) | 523 (16.2%) | 3020 (11.5%) | 2366 (4.7%) | 482 (7.1%) | 2848 (5.1%) |
| Overall | 3103 | 1155 | 4258 | 2754 (−11.2%) | 939 (−18.7%) | 3693 (−13.3%) | 2627 (−15.3%) | 918 (−20.5%) | 3545 (−16.7%) |
| United States | 1404 | 580 | 1984 | 1197 (−14.7%) | 449 (−22.6%) | 1646 (−17%) | 1166 (−17%) | 439 (−24.3%) | 1605 (−19.1%) |
| Completed | |||||||||
| Europe | 808 | 401 | 1209 | 741 (−8.3%) | 338 (−15.7%) | 1079 (−10.8%) | 696 (−13.9%) | 329 (−18%) | 1025 (−15.2%) |
| Asia | 555 | 147 | 702 | 433 (−22%) | 138 (−6.1%) | 571 (−18.7%) | 403 (−27.4%) | 136 (−7.5%) | 539 (−23.2%) |
| Primary Outcome Completed | |||||||||
| Overall | 3049 | 1109 | 4158 | 2484 (−18.5%) | 900 (−18.8%) | 3384 (−18.6%) | 2336 (−23.4%) | 870 (−21.6%) | 3206 (−22.9%) |
| United States | 1425 | 552 | 1977 | 1108 (−22.2%) | 422 (−23.6%) | 1530 (−22.6%) | 1071 (−24.8%) | 407 (−26.3%) | 1478 (−25.2%) |
| Europe | 818 | 399 | 1217 | 654 (−20%) | 316 (−20.8%) | 970 (−20.3%) | 600 (−26.7%) | 305 (−23.6%) | 905 (−25.6%) |
| Asia | 467 | 136 | 603 | 412 (−11.8%) | 149 (9.6%) | 561 (−7%) | 381 (−18.4%) | 145 (6.6%) | 526 (−12.8%) |
Abbreviations: COVID, coronavirus disease; Hosp, hospital; Pharma, pharmaceutical.
We also summarize the effects of COVID‐19 for individual countries (based on the primary location of the clinical trial source), provided the given country had at least 5 submitted/completed clinical trials between April 1 and October 31, 2019. The percent changes are graphed on a world map in Figure 3 using the R package maps. 11 There is some country‐by‐country variation in the number of submitted clinical trials, but the differences are not dramatic. Most countries witness a decrease in the number of completed clinical trials in 2020, although a handful of countries remained resilient. The most notable country is Egypt, which had a 69% increase in the number of submitted trials (from 435 to 734 trials) and a 73% increase in the number of completed trials (from 105 to 182). This dramatic increase is presumably in response to a 2018 Egyptian parliamentary bill governing clinical medical research and subsequently approved an amended version in August of 2020. 12
FIGURE 3.
Percent changes in the number of submitted and completed clinical trials from April through October of 2019 to April through October of 2020. NA, not applicable
Figure 4 depicts the longitudinal seasonally adjusted log monthly count data of the number of submitted clinical trials (top row), completed clinical trials (middle row), and primary outcome completed trials (bottom row) annotated with a linear model fit and confidence bands following an interrupted time series linear model. 10 The p values presented in the graphic correspond to an analysis of variance of model 1 compared to model 2 (presented in the Methods section). The analyses presented in Figure 3 largely mimic the descriptive statistics presented in Table 1 and Figure 3: COVID‐19 did not seem to have much impact on the number of submissions, but there is a significant impact on completed clinical trials and trials whose primary outcome was completed. However, Figure 4 differs with Table 1 in the presentation of a statistically significant increase in the number of clinical trial submissions from US sources (p = 0.029). This increase is largely due to COVID‐19‐specific trials; 472 post‐COVID US trial submissions (11% of the total) were identified by CTTI as COVID‐19 related clinical studies.
FIGURE 4.
Seasonally adjusted log monthly count data of the number of submitted and completed clinical trials by region
In Figure 5 we compared the number of submissions based on whether the source is an academic medical center/hospital/non‐profit foundation (simply labeled as “Academic”) versus a pharmaceutical/therapeutic/biotech company (simply labeled as “Pharma”). This analysis shows pharma submissions were significantly increased in the post‐COVID era (p = 0.0047) whereas no significant effects of COVID‐19 were presented among academic submissions (p = 0.82). The analysis also indicates a significant negative impact of COVID‐19 on completed clinical trials for pharmaceutical companies (p = 0.042), although COVID‐19 had a much more dramatic effect on clinical trials with an academic source (p < 0.0001). Similar results hold for primary‐outcome‐completed clinical trials.
FIGURE 5.
Seasonally adjusted log monthly count data of the number of submitted and completed clinical trials by sector
In Table 2, we model the seasonally adjusted log monthly counts of submitted and completed trials against the variables of time, pre‐post COVID indicator, sector, region, and an interaction of the COVID indicator with sector. This model largely reiterates the results previously stated: COVID‐19 has not had a significant impact on submitted clinical trials, but it has had a significant impact on completed clinical trials. In particular, COVID‐19 has had the most dramatic effect on clinical trial completions from an academic medical center.
TABLE 2.
Results of multiple linear models of seasonally adjusted log monthly counts of submitted and completed trials
| Characteristic | Submitted | Completed | Primary outcome completed | ||||||
|---|---|---|---|---|---|---|---|---|---|
| Beta | 95% CI | p value | Beta | 95% CI | p value | Beta | 95% CI | p value | |
| 2019 baseline | 0.22 | 0.07, 0.36 | 0.003 | 0.30 | 0.21, 0.40 | <0.001 | 0.33 | 0.23, 0.43 | <0.001 |
| Time | 0.01 | −0.02, 0.03 | 0.6 | −0.03 | −0.05, −0.01 | <0.001 | −0.03 | −0.05, −0.01 | 0.014 |
| COVID indicator | −0.03 | −0.10, 0.05 | 0.5 | −0.34 | −0.42, −0.27 | <0.001 | −0.37 | −0.45, −0.29 | <0.001 |
| Pharma | — | — | — | — | — | — | |||
| Academic | 0.00 | −0.04, 0.04 | >0.9 | 0.00 | −0.03, 0.03 | >0.9 | 0.00 | −0.03, 0.03 | >0.9 |
| United States | — | — | — | — | — | — | |||
| Asia | 0.04 | 0.00, 0.09 | 0.045 | −0.01 | −0.04, 0.03 | 0.7 | 0.00 | −0.04, 0.04 | 0.8 |
| Europe | 0.01 | −0.04, 0.05 | 0.8 | 0.01 | −0.03, 0.05 | 0.6 | 0.00 | −0.04, 0.04 | >0.9 |
| COVID* Academic | 0.03 | −0.07, 0.12 | 0.5 | −0.15 | −0.23, −0.07 | <0.001 | −0.23 | −0.31, −0.14 | <0.001 |
Abbreviations: CI, confidence interval; COVID, coronavirus disease; Pharma, pharmaceutical.
Bolded values represent p‐values that are less than 0.05.
DISCUSSION
We did not find a significant impact of COVID‐19 on the number of submitted clinical trials, although there were some variations by region and country. In particular, COVID‐19‐related submissions have contributed to a statistically significant increase in the number of clinical trial submissions in the United States. However, COVID‐19 has hampered the completion of clinical trials almost universally. Our analysis was based on the clinical trials registered on ClinicalTrials.gov, which may not present an accurate assessment of the true number of clinical trials, as some individual trials could be missing from the database or certain study information could be missing from the record. 13 There could also be potential disparities in reporting across different geographical locations. We also note that, although each trial is associated with a given country based on its source location, some trials are conducted in other countries. Even with these limitations, ClinicalTrials.gov still provides valuable insight into the impact of COVID‐19 on submitted and completed clinical trials. Egypt’s passing of a bill regulating clinical medical research appears to have caused it to buck the usual trend, as that country is associated with an impressive number of recently submitted and completed clinical trials. We also show that pharmaceutical/biotech/therapeutic trials have been more resilient to the effects of COVID‐19 compared with academic medical center/hospital/foundation trials. Finally, as the COVID‐19 pandemic is still on‐going, its long‐term effects on clinical trial research are still being realized.
CONFLICT OF INTEREST
The authors declared no competing interests for this work.
AUTHOR CONTRIBUTIONS
N.H. and A.B. wrote the manuscript, designed the research, performed the research, and analyzed the data.
Supporting information
Supplementary Material
Supplementary Material
Supplementary Material
Funding information
The project described was partially supported by the National Center for Advancing Translational Sciences, National Institutes of Health (NIH), through Grant UL1 TR002014. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.
REFERENCES
- 1. Cutler DM, Summers LH. The COVID‐19 pandemic and the $16 trillion virus. Am Econ J Econ Policy. 2020;324:1495‐1496. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Medidata . COVID‐19 and clinical trials: the Medidata perspective. Release 5.0. (2020). https://www.medidate.com/en/insight/covid‐19‐and‐clinical‐trials‐the‐medidata‐perspective.
- 3. Xue JZ, Smietana K, Poda P, Webster K, Yang G, Agrawal G. Clinical trial recovery from COVID‐19 disruption. Nat Rev Drug Discov. 2020;19:662‐663. [DOI] [PubMed] [Google Scholar]
- 4. van Dorn A. COVID‐19 and readjusting clinical trials. Lancet (London, England). 2020;396:523‐524. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Asaad M, Habibullah NK, Butler CE. The impact of COVID‐19 on clinical trials. Ann Surg. 2020;272:e222‐e223. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Bobato A, Gongora L, Leonardo D, Jardim F, Bastos DA. Oncology clinical trials during the COVID‐19 pandemic. Oncology. 2020;34:265‐269. [PubMed] [Google Scholar]
- 7. Kent DG, Knapp DJHF, Kannan N. Survey says: “COVID‐19 lockdown hits young faculty and clinical trials”. Stem Cell Reports. 2020;15:1‐5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Bhaskaran K, Gasparrini A, Hajat S, Smeeth L, Armstrong B. Time series regression studies in environmental epidemiology. Int J Epidemiol. 2013;42:1187‐1195. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Hyndman RJ, Khandakar Y. Automatic time series forecasting: the forecast package for R. J Stat Softw. 2008;27:22. [Google Scholar]
- 10. Bernal JL, Cummins S, Gasparrini A. Interrupted time series regression for the evaluation of public health interventions: a tutorial. Int J Epidemiol. 2017;46:348‐355. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Becker RA, Wilks AR, Brownrigg R, Minka TP, Deckmyn A. Maps: draw geographical maps. (2018).
- 12. Gomaa A. How this law can help Egypt in clinical trials amid COVID‐19 crisis. Al‐Monitor. 2020.
- 13. Tse T, Fain KM, Zarin DA. How to avoid common problems when using ClinicalTrials.gov in research: 10 issues to consider. BMJ. 2018;361:k1452. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Supplementary Material
Supplementary Material
Supplementary Material