Use of connected digital products in clinical research following the COVID-19 pandemic: a comprehensive analysis of clinical trials

Abstract

Objectives

In an effort to mitigate COVID-19 related challenges for clinical research, the US Food and Drug Administration (FDA) issued new guidance for the conduct of ‘virtual’ clinical trials in late March 2020. This study documents trends in the use of connected digital products (CDPs), tools that enable remote patient monitoring and telehealth consultation, in clinical trials both before and after the onset of the pandemic.

Design

We applied a comprehensive text search algorithm to clinical trial registry data to identify trials that use CDPs for remote monitoring or telehealth. We compared CDP use in the months before and after the issuance of FDA guidance facilitating virtual clinical trials.

Setting

All trials registered on ClinicalTrials.gov with start dates from May 2019 through February 2021.

Outcome measures

The primary outcome measure was the overall percentage of CDP use in clinical trials started in the 10 months prior to the pandemic onset (May 2019–February 2020) compared with the 10 months following (May 2020–February 2021). Secondary outcome measures included CDP usage by trial type (interventional, observational), funder type (industry, non-industry) and diagnoses (COVID-19 or non-COVID-19 participants).

Results

CDP usage in clinical trials increased by only 1.65 percentage points, from 14.19% (n=23 473) of all trials initiated in the 10 months prior to the pandemic onset to 15.84% (n=26 009) of those started in the 10 months following (p<0.01). The increase occurred primarily in observational studies and non-industry funded trials and was driven entirely by CDP usage in trials for COVID-19.

Conclusions

These findings suggest that in the short-term, new options created by regulatory guidance to stimulate telehealth and remote monitoring were not widely incorporated into clinical research. In the months immediately following the pandemic onset, CDP adoption increased primarily in observational and non-industry funded studies where virtual protocols are likely medically necessary due to the participants’ COVID-19 diagnosis.

Strengths and limitations of this study.

• This study is the first to quantify postpandemic trends in the use of telehealth and remote patient monitoring technology within the clinical research setting.

• A comprehensive text search algorithm was created and used to identify telehealth and remote monitoring use in clinical trials.

• Detailed, manual review of 500 trial records validated the search term list and algorithm, confirming a high degree of sensitivity and specificity.

• The limited postperiod analysis window (10 months) precludes the ability to draw implications for the pandemic’s longer-term impact on the conduct of clinical research.

• The identification of connected digital products in clinical trials is dependent on the trial sponsor entering information about remote monitoring and/or telehealth delivery when creating or updating the ClinicalTrials.gov record.

Introduction

When the WHO declared COVID-19 a global pandemic in March 2020, many aspects of healthcare were rapidly upended including the traditional conduct of clinical research. Preliminary evidence has documented the suspension of critical clinical trials and postponement of new trial start dates due to enrolment challenges and site closures.1–3

In an effort to keep clinical research moving forward at a time when visits to in-person sites were challenging and risky, the US Food and Drug Administration (FDA) issued new, comprehensive guidance for the conduct of clinical trials, with the aim of encouraging virtual patient consultations and remote collection of clinical outcomes assessments.4

Connected digital products (CDPs) are portable software and sensor-based technologies that are designed for patient use with little to no clinician involvement. These products collect health-related measurements from patients and thus enable remote monitoring and virtual consultation in the clinical research setting. Examples of CDPs include wearable activity trackers and heart rate monitors, mobile apps used for data delivery and collection, ingestible sensors, and health assessments conducted via a mobile platform.5 In the years prior to COVID-19, a dramatic increase in the use of CDPs has been documented in clinical trials.5 Though the appeal of CDPs is clear6 7—both for the pandemic duration and beyond—evidence describing their adoption in the clinical research setting following COVID-19 is lacking.

In an effort to understand recent implementation of remote monitoring and telehealth in clinical research, this study asks whether CDP use in clinical trials increased following the FDA’s March 2020 issuance of guidance for the conduct of virtual trials. The study documents trends in the use of CDPs in clinical trials that started both before and after the emergence of pandemic-related challenges for traditional protocols. Trends in CDP adoption data within the clinical research setting are particularly relevant in light of the pandemic-induced broader telehealth-related policy changes, such as the rapid movement of payers to cover a broad range of virtual care delivery services and particular generosity in the coverage of remote patient monitoring technologies.^{8 9}

Methods

We applied a text search algorithm to the full set of trial records downloaded from ClinicalTrials.gov to identify trials that used CDPs.10 ClinicalTrials.gov is a publicly available database that is considered comprehensive, as trial registration is required (1) when a study is being used to support the regulatory approval of a new therapeutic product and/or (2) when it is intended for publication in one of the International Committee of Medical Journal Editors’ member journals.11 12 To conduct the search, we adapted the set of CDP search terms published in Marra et al5 to include additional terms specific to remote monitoring and telehealth delivery (online supplemental file includes the list of search terms). We used an automated algorithm to search for each term in several relevant ClinicalTrials.gov fields, such as “official title”, “brief summary”, “detailed description”, “interventions”, “study arms” and “outcome measures”. We validated our search protocol with a manual review of 500 randomly selected trials (table 1 includes examples of qualified CDP trials).

Table 1.

Examples of connected digital product usage identified in clinical trials

CDP search term(s)	Trial type	Trial summary	ClinicalTrials.gov identifier
Fitbit, video-conferencing	Preperiod, non-COVID, interventional, non-industry	Hospital-sponsored, phase 4 trial to determine whether an approved drug for attention deficit disorder can help patients with mild cognitive impairment. Fitbit Charge 3 is used to continuously monitor sleep and activity. All visits occur over videoconferencing.	NCT03811847
Home-based pulse oximeter, virtual care platform	Postperiod, COVID, interventional, non-industry	Canadian research institute-led trial to demonstrate effectiveness and efficacy of a virtual care model for patients with COVID-19 in home isolation. Patients are given a home-based pulse oximeter and access to a virtual care platform (VIRUTES).	NCT04420182
Apple watch	Postperiod, non-COVID, interventional, non-industry	University-sponsored interventional trial testing whether a novel intervention reduces early reoccurrence of atrial fibrillation after catheter-based ablation. Uses Apple Watch worn continuously by participants to measure and record ECG data for the primary endpoint.	NCT04433091
Telemedicine, eDiary	Postperiod, non-COVID, interventional, industry	Pfizer-led phase 2a proof of concept study evaluating the efficacy and safety of crisaborole in adults with stasis dermatitis. Uses a de-centralised trial design involving remote contact by telemedicine and an eDiary for participant data collection.	NCT04091087
Televisit	Postperiod, non-COVID, interventional, non-industry	VA-sponsored interventional trial evaluating the effect of a pharmacist led medication management intervention on improving medication use for elderly patients with complex medication routines. Interactions occur via televisit.	NCT04340570

Source: ClinicalTrials.gov.

Our sample period includes trials with start dates between 1 May 2019 and 28 February 2021. We divided the first 10 months of the sample into prepandemic onset (‘preperiod’) and the last 10 months of the sample into postpandemic onset (‘postperiod’). We excluded trials with start dates in March and April of 2020, as FDA guidance on the conduct of clinical trials and other public policies encouraging telehealth use were introduced during this time, and trial sponsors would have needed this window to make protocol adjustments in response to new policies (table 2 presents a brief timeline of key events). For analysis purposes, trials were segmented into COVID-19 or non-COVID trials based on diagnoses listed in the ‘conditions’ field. Trial type (observational or interventional) and funder type (industry or non-industry) were collected from standardised ClinicalTrials.gov fields associated with each trial.13 To test whether the proportion of trials using CDPs was statistically different in the preperiod and postperiod, we performed two-sided z-tests using the means and standard deviation from the 10-month observation periods in each group. The analysis was conducted using STATA V.15.

Table 2.

Timeline of selected COVID-19 related regulatory guidance and policy changes encouraging the use of telehealth and remote monitoring in the USA

Date	Policy/event name	Description
11 March 2020	WHO declares COVID-19 a pandemic16
16 March 2020	Emergency declarations issued in every US state17	Acting under emergency orders, many state governors enact stay-at-home orders requiring individuals to remain in their residence, shutting down non-essential in-person commerce and requiring postponement of elective medical procedures.
17 March 2020 (revised 30 April, 25 June)	COVID-19 Emergency Declaration Blanket Waivers for Health Care Providers18	CMS issues temporary emergency order to expand Medicare coverage of telehealth services provided by licensed providers, including doctors, nurse practitioners and physician assistants.
20 March 2020 (revised 5 June)	Enforcement Policy for Non-Invasive Remote Monitoring Devices Used to Support Patient Monitoring During the COVID-19 Public Health Emergency19	FDA releases first guidance for industry and FDA staff encouraging the use of certain non-invasive remote monitoring devices to support patient monitoring during COVID-19 by suggesting the agency will not object to limited modifications to the indications, claims, functionality, or hardware or software.
27 March 2020 (revised 2 July, 21 September)	FDA Guidance on Conduct of Clinical Trials of Medical Products during COVID-19 Public Health Emergency20	FDA releases first guidance for industry, investigators, and institutional review boards on conduct of clinical trials, suggesting trial sponsors consider using telephone and video consultation in clinical trials to replace in-person site visits and collect clinical outcome assessments remotely using validated technology when appropriate.
30 April–15 May 2020	‘Stay at home’ orders lifted for most US states21	Majority of states lift their ‘stay at home’ orders but encourage individuals to limit unnecessary trips outside the home. Phased reopening begins and varies by state.

Sources: (a) Timeline: WHO16 (b) Kaiser Family Foundation17 (c) Centers for Medicare & Medicaid Services18 (d) FDA.gov19 (e) FDA.gov20 (f) Moreland A et al.21

Patient and public involvement statement

Neither patients nor the public were involved in this research.

Results

Over the 22-month period from May 2019 to February 2021, the overall use of CDPs in clinical trials remained relatively stable, accounting for between 12% and 19% of all newly started trials in each month of our dataset. Trials using CDPs accounted for 14.19% (3330/23 473) of all trials started in the preperiod and 15.84% (4121/26 009) of trials started in the postperiod. This 1.65 percentage point increase represents a small but statistically significant uptick in CDP usage for trials with start dates after the pandemic onset (p<0.01) (figure 1).

Figure 1.

Proportion of clinical trials started monthly that use connected digital products for telehealth delivery or remote monitoring. The preperiod and postperiod analysis windows include the 10 months before and after the COVID-19 pandemic onset. Percentages on the left and right represent the average percentage of trials using connected digital products during the preperiod and postperiod, respectively. *Statistical significance in the mean difference between the preperiods and postperiods using two-sided proportion tests. Source: authors’ analysis of ClinicalTrials.gov data.

Among trials labelled as observational, there was a larger average increase in trials using CDPs from 11.7% (569/4863) preperiod to 14.6% (799/5490) postperiod (p<0.001). There was also an increase in the percentage of trials using CDPs among studies funded by non-industry parties, such as hospitals, academic institutions and government (16.2%–18.3%, p<0.01). The use of CDPs in industry-funded trials was notably lower than the sample mean both before and after the FDA’s guidance was issued in response to the pandemic onset (7.9% vs 8.9%, p=0.17), although this share also experienced a small overall increase, on average.

Trials with start dates in the postperiod were segmented into COVID-19 (n=3294) and non-COVID (n=22 715) diagnoses. Among trials conducted in patients with COVID-19, use of CDPs was significantly higher than the preperiod average (20.0% vs 14.2%, p<0.0001). However, among trials unrelated to COVID-19 (‘non-COVID’) that were started in the postperiod, the change in CDP usage compared with the preperiod was very small and not statistically significant (15.0% vs 14.2%, p=0.19). As such, the observed increase in CDP usage among observational and non-industry funded trials appears to have been driven by studies related directly to COVID-19 (figure 2).

Figure 2.

Proportion of clinical trials started that use connected digital products for telehealth delivery or remote monitoring, by study type and funder type. ‘Preperiod’ and ‘postperiod’ refer to the 10 months immediately before and after COVID-19 onset in the USA. March and April 2020 are excluded because policy changes and guidance documents were issued during that time. Trials started in the postperiod are divided into ‘COVID-19’ trials and ‘non-COVID’ trials based on their diagnoses in the ClinicalTrials.gov conditions label. *Statistical significance in the mean difference between the preperiods and postperiods using two-sided proportion tests. Source: authors’ analysis of ClinicalTrials.gov data.

To further validate the results, we performed a series of robustness checks. First, we removed trials where the recruitment status suggested that the study had not begun recruiting participants even though the start date had technically passed.14 Within the subset of trials where commencement of enrolment had been confirmed (n=19 785 preperiod, 18 426 postperiod), there was no statistically significant difference in overall CDP usage between the preperiod and postperiod (15.1%–15.9%, p=0.38). Differences seen in observational and COVID-specific trials were similar to those observed in the overall sample. Second, we removed trials conducted outside of the USA because the guidance encouraging virtual trials was issued by the US FDA. Though we find a somewhat higher rate of CDP adoption among US trials (between 15.7% and 24.3% per month), the average increase in CDP usage among US trials following the pandemic onset was the same as the average seen for all trials in the sample (1.7 percentage points from preperiod to postperiod) (online supplemental file).

Discussion

Overall, the use of CDPs in clinical trials increased only very modestly in trials with start dates occurring after COVID-19 was declared a global pandemic. This finding is surprising in light of new regulatory guidance introduced in late March 2020, which aimed to bolster the use of CDPs to sustain clinical research. Moreover, these modest increases stand in sharp contrast to the multiple-order-of-magnitude increases in the use of telemedicine for routine healthcare delivery that were observed over the same period of time.15 Though it may still be too early to observe the full effect of new regulatory initiatives, our findings suggest that the growing enthusiasm for telehealth and remote patient monitoring in mitigating pandemic-related healthcare challenges has yet to be realised in the clinical research setting.

There are several possible explanations for why CDP usage in the clinical research setting did not increase as dramatically as expected in the months following the pandemic onset. On one hand, the increase in CDP usage documented among observational trials (rather than interventional trials) suggests the relative ease of incorporating new remote monitoring technology into studies that do not evaluate therapeutic benefit. Implementation of protocol changes may be more accessible in the context of observational analysis that does not require rigorous prior planning and approval to incorporate into a study design. On the other hand, the consistently lower adoption of CDPs by industry funders reinforces that firms may continue to perceive substantial risk associated with reliance on technology for trials that require regulatory review or may be reluctant to augment trial protocols and operations—even after the FDA explicitly encouraged protocol modifications to enable virtual trials. Further, the extent to which FDA guidance should elicit a change in protocol design by trial sponsors should be most acutely observed in trials with US sites. However, we find the same percentage point increase in CDP usage following the pandemic onset for US trials as was seen in all trials. It is also possible that clinical trial investigators do not yet see value in adding CDPs to their trials or lack the technical expertise to do so, despite regulatory encouragement.

Additionally, the fact that COVID-specific trials drove the observed increase in CDP usage suggests that the changes in FDA guidance may have primarily impacted trials where virtual protocols were most necessary for the health and safety of clinicians and other patients (in this case, given the trial participants’ diagnosis). Though an increase in CDP usage among non-COVID related trials may still materialise in the future, there has yet to be any evidence of massive take-up on the scale of other services, such as telemedicine. Though several factors could add to investigator and trial sponsor hesitancy in the adoption of remote monitoring and telehealth technology, the findings of this study suggest that trial sponsors may be unwilling or unable to adopt CDPs in their research design until there is either medical necessity or formalised regulatory requirements to do so.

Finally, robustness checks find a very small but non-statistically significant increase in CDP usage among the subset of trials where commencement of participant recruitment could be confirmed, suggesting that at least some of the small overall increase in CDP usage may be attributable to trials that were planned to start within the postperiod analysis window (1 May 2020–28 February 2021) but may have experienced delays.

The study’s primary limitation is its somewhat short analysis period. The trends identified are only representative of CDP usage in the 10 months immediately following issuance of the FDA’s guidance for conducting virtual protocols (May 2020–February 2021) and implications for long-term trends cannot yet be drawn. Other limitations include reliance on data entered by trial sponsors and a risk of incomplete data since trials can be retrospectively registered after their start date. We have attempted to mitigate the latter issue by waiting 5 weeks beyond the end of the analysis period to download records from ClinicalTrials.gov.

Conclusion

Though the pandemic may lead to sustained changes in how healthcare is delivered, the longer-term implications for the use of connected technology in clinical research are less clear. Despite regulatory guidance encouraging remote monitoring and telehealth use in clinical trials, we did not find a meaningful increase in the usage of CDPs in trials started after the pandemic onset. Trends in the use of CDPs will be important to monitor as COVID-19 continues to impact investigators’ ability to conduct clinical research and further policies around telehealth are introduced.

Data availability statement

Data are available in a public, open access repository. At the time of publication, the authors will share data from the study on the public GitHub repository https://github.com/arieldora/Covid19digitaltrials.

Ethics statements

Patient consent for publication

Not required.

Ethics approval

This study analyses publicly available clinical trial protocols that have been registered in the freely accessible ClinicalTrials.gov database provided by the National Library of Medicine.