Abstract
Background
Internet‐based participation has the potential to enhance pragmatic and decentralized trials, where representative study populations and generalizability to clinical practice are key. We aimed to study the differences between internet and noninternet/telephone participants in a large remote, pragmatic trial.
Methods and Results
In a subanalysis of the ADAPTABLE (Aspirin Dosing: A Patient‐Centric Trial Assessing Benefits and Long‐Term Effectiveness) study, we compared internet participants with those who opted for noninternet participation. Study process measures examined included participant characteristics at consent, study medication adherence, and study retention. The clinical outcome examined was a composite of all‐cause mortality, hospitalization for myocardial infarction, or hospitalization for stroke. Noninternet participants were older (mean 69.4 versus 67.4 years), more likely to be female (38.9% versus 30.2%), more likely to be Black (27.3% versus 6.0%) or Hispanic (11.1% versus 2.0%), and had a higher number of comorbid conditions. The composite clinical outcome was more than twice as high in noninternet participants. The hazard of nonadherence to the assigned aspirin dosage was 46% higher in noninternet participants than internet participants.
Conclusions
Noninternet participants differed from internet participants in notable demographic characteristics while having poorer baseline health. Over the course of ADAPTABLE, they also had worse clinical outcomes and greater likelihood of study drug nonadherence. These results suggest that trials focused on internet participation select for younger, healthier participants with a higher proportion of traditionally overrepresented patients. Allowing noninternet participation enhances diversity; however, additional steps may be needed to promote study retention and study medication adherence.
Registration Information
clinicaltrials.gov. Identifier: NCT02697916.
Keywords: cardiovascular disease, clinical trial methodology, internet follow‐up, pragmatic trials
Subject Categories: Vascular Disease, Clinical Studies
Nonstandard abbreviations and acronyms
- ADAPTABLE
Aspirin Dosing: A Patient‐Centric Trial Assessing Benefits and Long‐Term Effectiveness
- GEE
generalized estimating equation
Clinical Perspective
What Is New?
The effects of internet‐based methods of participation on clinical trial outcomes and representativeness of study participants is largely unknown.
As a large‐scale, decentralized, pragmatic clinical trial, the ADAPTABLE (Aspirin Dosing: A Patient‐Centric Trial Assessing Benefits and Long‐Term Effectiveness) study provided a context in which to compare participants who opted for internet participation to those who opted for noninternet participation.
ADAPTABLE's noninternet participants tended to be from traditionally underrepresented demographic groups and had worse clinical outcomes.
What Are the Clinical Implications?
Caution may be warranted in using Internet participation in cardiovascular studies, considering its effects on participant diversity and statistical power.
The internet has become a critical tool in the clinical research arsenal, particularly in the administration of large, multicenter trials. As modern clinical trials grow in complexity, the internet has helped provide resources for protocol development; aided in communication among trial personnel; helped recruit, register, consent, and randomize patients; and facilitated data entry, analysis, and validation. 1 , 2 , 3 The internet is also increasingly used as a platform for direct patient participation, facilitating the performance of decentralized and pragmatic trials, and introducing process and cost efficiencies. 4 Given its ability to reach large, geographically diverse populations, this latter form of internet usage in clinical trials continues to increase in the setting of the recent COVID‐19 pandemic. 5 , 6 , 7 , 8
Pragmatic trials assess the effectiveness of interventions under usual clinical conditions, with an overall aim of producing generalizable results to inform clinical decisions in typical practice settings. 9 , 10 A key component of pragmatic trials is the inclusion of participants representative of patients eligible for the intervention in routine clinical care. 11 Although internet use offers potential benefits, studies on digital literacy and nationwide broadband internet access have noted disparities adversely affecting populations already at risk for underrepresentation in cardiovascular clinical trials, with almost 1 in 5 seniors and 1 in 5 Americans living in poverty lacking internet access. 12 Digital literacy rates are also lower in senior, Black, and Hispanic people, who are more likely to report no internet use in their daily lives or issues with access. 13 , 14 , 15 The ADAPTABLE (Aspirin Dosing: A Patient‐Centric Trial Assessing Benefits and Long‐term Effectiveness) study offers an opportunity to glean contemporary insight into large‐scale, pragmatic studies with internet follow‐up within the context of cardiovascular clinical trials. With the inclusion of a noninternet follow‐up arm, ADAPTABLE facilitates comparisons between internet and noninternet participants. In this study, we compare baseline characteristics, clinical outcomes, study medication adherence, and visit completion outcomes among participants with and without internet participation.
Methods
The data that support the findings of this study are available from the corresponding author upon reasonable request.
Study Design and Population
The trial methods and primary results of ADAPTABLE (NCT02697916) have previously been published. 16 , 17 ADAPTABLE was a large‐scale, decentralized, open‐label randomized controlled trial comparing the effectiveness and safety of an aspirin dose of 81 mg a day with 325 mg a day for the prevention of cardiovascular events or death in patients with established atherosclerotic cardiovascular disease. Participants were recruited to the study via email, letter, telephone, or face to face in a clinical setting. This study was approved by an institutional review committee and subjects gave informed consent.
Internet Versus Noninternet Participation
Two methods of follow‐up, internet and noninternet, were available for participants to choose at the beginning of the study. Internet visits entailed email reminders to complete the visit through an online patient portal, whereas noninternet participation included telephone call reminders to complete study visits with the Duke Clinical Research Institute Call Center. Initially, noninternet (call center) follow‐up was to be capped at 1000 participants, primarily due to cost considerations. However, after it became apparent that more potential participants preferred this follow‐up method, the noninternet follow‐up cap was raised. Those participants that initially chose internet participation but missed more than 1 internet portal encounter were then contacted by the Duke Clinical Research Institute Call Center for a noninternet “rescue” follow‐up visit. Internet participants were able to go back and forth between internet and noninternet participation over the course of the study. For this study analysis, participants were grouped based on their initial method of participation, even if they crossed over to the other participation method at some point.
End Points
We considered both previously defined primary outcomes based on the main ADAPTABLE study alongside the main measure of interest specific to this subanalysis (ie, the underlying differences between internet and noninternet participants). The primary composite effectiveness end point from the main ADAPTABLE study was time from randomization to a composite of all‐cause death, hospitalization for myocardial infarction, or hospitalization for stroke. The primary safety outcome was hospitalization for major bleeding that was associated with blood transfusion. Secondary outcomes included the individual components of the primary composite, the occurrence of coronary revascularization, and patient‐reported quality of life. End points were ascertained via multiple sources including internet or noninternet patient follow‐up, electronic heath record data, and public and private insurance claims data. Contact was made with participants every 3 or 6 months (follow‐up interval was also randomized). We classified a participant as adherent at a visit if they reported taking their assigned aspirin dose and non‐adherent if they either changed to a different dose of aspirin or stopped their aspirin entirely. We considered that they completed a visit if they provided any aspirin follow‐up data at that visit, whether through the participant portal or by telephone.
If a participant did not experience an effectiveness or safety outcome, they were censored at the earlier of study end date, date of withdrawal of consent, death (nonfatal outcomes) or maximum follow‐up time point determined from the electronic heath record, insurance claims, or the patient portal (last point of contact). If a participant did not experience an adherence outcome, they were censored at the earlier of death, withdrawal of consent, or the last patient portal visit without the use of electronic heath record or claims data.
Statistical Analysis
Participants were grouped by choice of internet and noninternet participation at the time of randomization. Baseline characteristics were described using medians (interquartile ranges) and counts (percentages) as relevant. Continuous variables were compared using t tests (unless otherwise noted) and categorical variables using chi‐square tests. Additionally, baseline characteristics and follow‐up completion were also presented by method of participation at randomization broken down by method of participation at last observed visit.
To assess differences in clinical outcomes between participation methods (ie, internet versus noninternet), event rates were calculated among noninternet and internet participants using the cumulative incidence function estimator at median time of follow‐up (26.2 months) and as the number of events per 100 patient years of follow‐up. A Cox proportional hazards model was used to evaluate the relationship between internet participation and the primary effectiveness end point and all‐cause death. The Fine‐Gray method was used to evaluate the primary safety end point, hospitalization for myocardial infarction, and hospitalization for stroke outcomes to account for the competing risk of death. Unadjusted analyses are presented to reflect a belief that choice of internet or noninternet participation is reflective of many demographic and clinical characteristics of these groups; any differences in clinical outcomes reflect differences in these characteristics. To assess differences in clinical outcomes beyond that explained by potential confounders captured in ADAPTABLE, we also present the comparison adjusted for age, sex, race, ethnicity, and invitation method.
The proportional hazards assumption was evaluated using weighted Schoenfeld residuals. Event counts, cumulative event rates estimated at median follow‐up, incidence rates, hazard ratios (HRs) or subdistribution hazard ratios (95% CIs) comparing noninternet participants to internet participants and P values are presented.
Additionally, Cox proportional hazards models were used to evaluate the association between participation method and the composite adherence end point of aspirin discontinuation or dose switching and each of the individual components, aspirin discontinuation and dose switching. A cause‐specific Cox model taking into account the competing risk of aspirin discontinuation was used for the dose switching end point. The analysis was conducted both with and without adjustment for potential confounders. Adjustment variables were those judged to reasonably affect both medication adherence and internet use: age, sex, race, ethnicity, and invitation method. The proportional hazards assumption for internet participation was evaluated using weighted Schoenfeld residuals.
The interactions between aspirin dose and participation method with respect to the primary outcomes (effectiveness, safety, and adherence) were then assessed using the same models as described in prior paragraphs. Cumulative event rates estimated at median follow‐up were computed for each aspirin dose and participation method combination. HRs or cause‐specific HRs (CSHRs; 95% CI) comparing 81 mg to 325 mg were computed for both the noninternet and internet groups. The interaction P values are supplied.
The interaction between age and participation method was also assessed. Age was modeled using a natural cubic spline with knots at the 5th, 35th, 65th, and 95th percentiles.
Finally, completion of possible follow‐up visits was analyzed using a binomial logistic regression model fit with the generalized estimating equation method to model the probability that a patient would complete a possible visit. A given visit was defined to be “possible” for a patient if the patient's follow‐up interval group was assigned to complete the visit and the patient's date of death or end of study date are greater than the derived visit date. An autoregressive working correlation matrix was assumed. The model included noninternet/internet participation, days from randomization to the expected visit date (visit days), and the interaction between internet participation and visit days to allow the effect of internet participation to vary over the course of follow‐up. Both unadjusted and adjusted models were performed; adjustment variables were age, sex, race, ethnicity and invitation method. The relationship between visit days and visit completion was tested for linearity using natural cubic splines with knots at the 5th, 35th, 65th, and 95th percentiles. The linearity assumption was found to be violated and a piecewise linear spline with a single knot at 500 days was used for analysis. The odds ratio (OR; 95% CI) for noninternet versus internet participation was reported for the following visits: week 1, month 6, year 1, month 18, year 2, month 30, year 3, and month 42. The P value for the test of the interaction between internet participation and visit days (ie, the test of whether the effect of internet participation varied over the course of follow‐up) was also presented.
All analyses were conducted by the Duke Clinical Research Institute (Durham, NC) using SAS version 9.4 (SAS Institute, Inc, Cary, NC).
Results
Participants, Method of Follow‐Up, and Baseline Demographics
Recruitment began in April 2016 and ended in June 2019. Of the 15 076 participants enrolled and randomized to 81 or 325 mg aspirin dose, 13 172 participants (87.4%) initially chose to complete study encounters via the internet‐based patient portal. The remaining 1904 participants (12.6%) in the noninternet group chose to complete study encounters via the Duke Clinical Research Institute call center.
There were notable differences in many demographic characteristics between the 2 groups (Table 1). Noninternet participants were older with an average age of 69.4 (interquartile range 62.5–76.3), whereas the average internet participant was 67.4 years old (interquartile range 60.4–73.2). More noninternet participants were female (38.9% compared with only 30.2% of the internet participants, P≤0.001). The noninternet group was 27.3% Black compared with only 6.0% of the internet participant group (P<0.001), and 11.1% of noninternet participants were Hispanic versus 2.0% of internet participants (P<0.001). Noninternet participants were more likely to be current smokers (15.3% versus 8.9%, P<0.001).
Table 1.
Baseline Characteristics by Participation Method at Randomization
| Characteristic | Overall (N=15 076) | Noninternet participant (N=1904) | Internet participant (N=13 172) | P value |
|---|---|---|---|---|
| Age, y, median (Q1–Q3) | 67.6 (60.7–73.6) | 69.4 (62.5–76.3) | 67.4 (60.4–73.2) | <0.001 |
| Female | 4724 (31.3%) | 740 (38.9%) | 3984 (30.2%) | <0.001 |
| Race | <0.001 | |||
| White | 11 990 (79.5%) | 1045 (54.9%) | 10 945 (83.1%) | |
| Black | 1311 (8.7%) | 520 (27.3%) | 791 (6.0%) | |
| Asian | 146 (1.0%) | 26 (1.4%) | 120 (0.9%) | |
| American Indian or Alaska native | 114 (0.8%) | 26 (1.4%) | 88 (0.7%) | |
| Multiple/other | 535 (3.6%) | 217 (11.4%) | 318 (2.4%) | |
| Not reported | 980 (6.5%) | 70 (3.7%) | 910 (6.9%) | |
| Hispanic | 481 (3.2%) | 212 (11.1%) | 269 (2.0%) | <0.001 |
| Current smoker | 1382 (9.8%) | 289 (15.3%) | 1093 (8.9%) | <0.001 |
| Body mass index, kg/m2, median (Q1–Q3) | 30.0 (26.7–34.4) | 29.7 (26.3–34.0) | 30.1 (26.7–34.4) | 0.029 |
| Trial details | ||||
| Randomized dose | 0.174 | |||
| 81 mg | 7540 (50.0%) | 980 (51.5%) | 6560 (49.8%) | |
| 325 mg | 7536 (50.0%) | 924 (48.5%) | 6612 (50.2%) | |
| Randomized follow‐up interval | 0.451 | |||
| 3 months | 7541 (50.0%) | 937 (49.2%) | 6604 (50.1%) | |
| 6 months | 7535 (50.0%) | 967 (50.8%) | 6568 (49.9%) | |
| Invitation method | <0.001 | |||
| Received an email | 5900 (39.1%) | 53 (2.8%) | 5847 (44.4%) | |
| Received a letter | 3400 (22.6%) | 307 (16.1%) | 3093 (23.5%) | |
| Approached face to face in a clinical setting | 4080 (27.1%) | 1393 (73.2%) | 2687 (20.4%) | |
| Contacted by telephone | 1695 (11.2%) | 151 (7.9%) | 1544 (11.7%) | |
| Medical history* | ||||
| Prior myocardial infarction | 5305 (36.2%) | 889 (46.8%) | 4416 (34.6%) | <0.001 |
| Prior coronary artery bypass graft | 3527 (24.1%) | 547 (28.8%) | 2980 (23.4%) | <0.001 |
| Prior percutaneous coronary intervention | 5946 (40.6%) | 956 (50.3%) | 4990 (39.1%) | <0.001 |
| Cerebrovascular disease | 2624 (17.9%) | 474 (24.9%) | 2150 (16.8%) | <0.001 |
| Hypertension | 12 512 (85.3%) | 1761 (92.7%) | 10 751 (84.2%) | <0.001 |
| Hyperlipidemia | 12 946 (88.3%) | 1705 (89.7%) | 11 241 (88.1%) | 0.036 |
| Atrial fibrillation | 1233 (8.4%) | 162 (8.5%) | 1071 (8.4%) | 0.844 |
| Congestive heart failure | 3504 (23.9%) | 699 (36.8%) | 2805 (22.0%) | <0.001 |
| Peripheral artery disease | 3493 (23.8%) | 728 (38.3%) | 2765 (21.7%) | <0.001 |
| Diabetes | 5676 (38.7%) | 979 (51.5%) | 4697 (36.8%) | <0.001 |
| History of bleeding | 1267 (8.6%) | 235 (12.4%) | 1032 (8.1%) | <0.001 |
| Significant gastrointestinal bleed | 950 (6.5%) | 187 (9.8%) | 763 (6.0%) | <0.001 |
| Intracranial hemorrhage | 208 (1.4%) | 38 (2.0%) | 170 (1.3%) | 0.022 |
| Prior medications | ||||
| Prior aspirin use† | <0.001 | |||
| No use | 566 (4.0%) | 29 (1.5%) | 537 (4.4%) | |
| 81 mg | 11 547 (82.0%) | 1624 (86.1%) | 9923 (81.4%) | |
| 162 mg | 310 (2.2%) | 26 (1.4%) | 284 (2.3%) | |
| 325 mg | 1657 (11.8%) | 208 (11.0%) | 1449 (11.9%) | |
| P2Y12 inhibitor‡ | 3051 (22.1%) | 478 (25.4%) | 2573 (21.6%) | <0.001 |
Percentages are based on 14 661 participants with available medical history data (1900 noninternet and 12 761 internet participants).
Percentages are based on 14 080 participants with available aspirin history data (1887 noninternet and 12 193 internet participants).
Percentages are based on 13 818 participants with available medications data (1884 noninternet and 11 934 internet participants).
Noninternet participants had more comorbidities and a greater percentage of those who had undergone prior coronary revascularization procedures. Of the noninternet participants, 46.8% had a history of myocardial infarction, 36.8% had congestive heart failure, and 51.5% had diabetes compared with 34.6%, 22.0% and 36.8% respectively in internet participants (P<0.001 for all). Noninternet participants also had higher likelihood of prior bleeding, baseline P2Y12 inhibitor use, prior significant gastrointestinal bleeding, and prior intracranial hemorrhage than internet participants. Recruitment method also differed between the groups: 73.2% of noninternet participants were approached face to face in a clinic or hospital, versus only 20.4% of internet participants.
Clinical Outcomes
Estimated cumulative incidence at median follow‐up of the composite clinical end point of all‐cause death, hospitalization for myocardial infarction, or hospitalization for stroke was more than twice as high in noninternet participants (13.35% noninternet versus 6.49% internet; HR, 2.16 [95% CI, 1.99–2.47]; adjusted HR, 1.67 [95% CI, 1.42–1.97]; Table 2, Figure 1). The effect of aspirin dose on the primary effectiveness outcome was not significantly modified by internet participation at randomization (P=0.150 unadjusted, 0.108 adjusted) (Table S1). There was no significant interaction between internet participation and age with regard to the composite end point (P=0.575).
Table 2.
Association Between Internet Participation and Outcomes (Adherence and Clinical)
| Outcome | Cumulative incidence function estimate at median follow‐up | Incidence rate (events per 100 patient years of follow‐up) | Unadjusted | Adjusted* | ||||
|---|---|---|---|---|---|---|---|---|
| Noninternet participant | Internet participant | Noninternet participant | Internet participant | Hazard ratio (95% CI) | P value | Hazard ratio (95% CI) | P value | |
| Medication adherence end point: dose switching or aspirin discontinuation | 48.84 | 37.39 | 37.33 (788) | 24.25 (4171) | 1.44 (1.34–1.56) | <0.001 | 1.46 (1.34–1.60) | <0.001 |
| Dose switching | 36.51 | 23.81 | 27.71 (585) | 15.12 (2601) | 1.71† (1.57–1.87) | <0.001 | 1.54† (1.39–1.71) | <0.001 |
| Aspirin discontinuation | 16.89 | 15.46 | 9.24 (255) | 8.67 (1722) | 1.06 (0.93–1.21) | 0.359 | 1.25 (1.07–1.45) | 0.005 |
| Composite clinical end point: all‐cause death, myocardial infarction, or stroke | 13.35 | 6.49 | 6.74 (271) | 3.14 (888) | 2.16 (1.88–2.47) | <0.001 | 1.67 (1.42–1.97) | <0.001 |
| Myocardial infarction | 5.03 | 2.62 | 2.36 (96) | 1.21 (345) | 1.90‡ (1.51–2.38) | <0.001 | 1.44‡ (1.09–1.92) | 0.011 |
| Stroke | 2.40 | 1.08 | 1.12 (46) | 0.52 (148) | 2.12‡ (1.53–2.96) | <0.001 | 1.39‡ (0.92–2.11) | 0.117 |
| All‐cause death | 7.60 | 3.58 | 3.86 (165) | 1.74 (507) | 2.24 (1.88–2.67) | <0.001 | 1.90 (1.54–2.35) | <0.001 |
| Safety end point: major bleeding with associated blood product transfusion | 0.79 | 0.59 | 0.46 (19) | 0.27 (78) | 1.65‡ (1.00–2.73) | 0.049 | 1.08‡ (0.59–1.97) | 0.797 |
Adjusted for age, sex, race, ethnicity, and invitation method.
Cause‐specific hazard ratio.
Subdistribution hazard ratio.
Figure 1. Composite clinical end point: all‐cause death, myocardial infarction, or stroke.
Kaplan–Meier event curves for primary composite end point by internet participation at randomization. There was a steady separation in the composite clinical end point between internet and noninternet participants, with a greater rate of increase in the noninternet participant group.
Estimated cumulative incidence at median follow‐up of the safety end point of major bleeding associated with blood product transfusion was higher in noninternet participants, but not significantly so following adjustment (0.79% versus 0.59%; subdistribution HR, 1.65 [95% CI, 1.00–2.73]; adjusted subdistribution HR, 1.08 [95% CI, 0.59–1.97]; Table 2, Figure S1).
Study Drug Adherence
The hazard of nonadherence to the assigned aspirin dosage (through dose switching or discontinuation) was 44% higher in noninternet participants than internet participants in unadjusted analysis (CSHR, 1.44 [95% CI, 1.34–1.56]; Table 2). After adjustment for potential confounders, the hazard of nonadherence was 46% higher in noninternet participants (adjusted CSHR, 1.46 [95% CI, 1.34–1.60]; Table 2). Nonadherence could be primarily attributed to dose switching, and separation in adherence differences between internet and noninternet participants occurred early during the follow‐up period (Figure 2A).
Figure 2. Trial medication nonadherence.
Kaplan–Meier curves for medication adherence outcomes by internet participation at randomization. Early nonadherence through dose switching or aspirin discontinuation occurred in the 325 mg groups across all participants, though at a higher rate in the noninternet group (A). B, Noninternet participants had higher rates of adherence regardless of starting dose.
In internet participants, estimated cumulative incidence of nonadherence (dose switching or aspirin discontinuation) at time of median follow‐up was 55.4% in those randomized to 325 mg of aspirin compared with 18.6% in those randomized to 81 mg aspirin (adjusted CSHR, 0.28 [95% CI, 0.26–0.30]). In noninternet participants, estimated cumulative incidence of nonadherence was 73.1% versus 25.0% in 325 mg and 81 mg aspirin groups, respectively (adjusted CSHR, 0.23 [95% CI, 0.19–0.27]) (Table S1). There is a significant interaction between assigned dose at randomization and choice of internet participation, meaning there is evidence to suggest that the effect of randomized dose differs by choice of participation method (P=0.019; Table S1, Figure 2B). There was no significant interaction between participation method and age with study drug adherence (P=0.062).
Visit Completion Outcomes
The overall visit completion rates were similar between internet and noninternet participants (median: 88.9% versus 87.5%, P=0.08) but the association of internet participation with visit completion varied over the course of follow‐up (unadjusted and adjusted interaction P values <0.001). In the first year of study conduct, a higher proportion of noninternet participants completed visits at week 1 (94.9% noninternet versus 86.2% internet; adjusted OR, 4.19 [95% CI, 3.61–4.87]), month 6 (90.2% versus 81.4%; adjusted OR, 2.92 [95% CI, 2.60–3.29]), and year 1 (81.4% versus 75.1%; adjusted OR, 2.00 [95% CI, 1.79–2.24]). Internet participants completed a higher proportion of visits at year 2, though the difference was not significant after adjustment (60.2% versus 68.1%; unadjusted OR, 0.71 [95% CI, 0.64–0.78]; adjusted OR, 0.96 [95% CI, 0.86–1.07]). The difference was significant after adjustment at year 3 (40.2% versus 65.8%; adjusted OR, 0.47 [95% CI, 0.39–0.56]) (Table 3).
Table 3.
Association Between Internet Participation and Visit Completion During Follow‐Up
| Visit completion | Noninternet participant percentage | Internet participant percentage | P value* | P value |
|---|---|---|---|---|
| Percentage of visits completed† | 87.5 (64.3–100.0) | 88.9 (72.7–100.0) | 0.08 | … |
| Unadjusted odds ratio‡ (95% CI) | Adjusted§ odds ratio (95% CI) | |||||
|---|---|---|---|---|---|---|
| Study visit|| | <0.001 | <0.001 | ||||
| Week 1 | 94.9%¶ (1888) | 86.2% (12203) | 3.00 (2.61–3.44) | 4.19 (3.61–4.87) | ||
| Month 6 | 90.2% (1621) | 81.4% (10093) | 2.11 (1.89–2.34) | 2.92 (2.60–3.29) | ||
| Year 1 | 81.4% (1378) | 75.1% (8462) | 1.45 (1.32–1.61) | 2.00 (1.79–2.24) | ||
| Month 18 | 69.5% (1053) | 69.2% (6665) | 1.01 (0.91–1.12) | 1.38 (1.23–1.56) | ||
| Year 2 | 60.2% (651) | 68.1% (4567) | 0.71 (0.64–0.78) | 0.96 (0.86–1.07) | ||
| Month 30 | 50.2% (318) | 67.0% (2841) | 0.50 (0.44–0.56) | 0.67 (0.58–0.77) | ||
| Year 3 | 40.2% (55) | 65.8% (935) | 0.35 (0.29–0.42) | 0.47 (0.39–0.56) | ||
| Month 42 | 31.0% (1) | 64.7% (138) | 0.24 (0.19–0.31) | 0.32 (0.25–0.42) |
The P value is from a test of the interaction between internet participation and visit days (ie, the test of whether the effect of internet participation varied over the course of follow‐up).
Median (Q1–Q3) % completed; P value comes from the nonparametric Wilcoxon rank‐sum test comparing distributions between groups.
The odds ratio and 95% CI reflect the odds of completing a given study visit for noninternet participants compared with internet participants.
Adjusted model includes age, sex, race, ethnicity, and invitation method.
A generalized estimating equation model was fit with internet participation, visit days, and their interaction. Visit days were nonlinear with respect to visit completion; therefore, 2 linear splines (knot@500 days) were used.
Percentages are model‐based predicted probabilities. Events are the number of completed visits at the specified visit time.
Of note, 1698 of the initial 13 172 (12.9%) internet participants switched to noninternet participation as of their last visit whereas 13 of the 1904 (0.68%) initial noninternet participants completed their final visit via internet (Table S2). There was a marked drop in individual 100% study completion rate (ie, completing all study visits) in participants who started as internet participants but completed the study as noninternet participants (20.3% completed all visits compared with 49.8% for those who started and finished as internet participants and 45.4% for all participants) (Table S2).
Discussion
In the ADAPTABLE study, internet and noninternet modes of participation were offered to promote patient‐centeredness in decentralized participation and key differences were found between participants who self‐selected into noninternet and internet participation groups. At baseline, noninternet participants were older, more likely to be female, Black, Hispanic, and more likely to have a higher burden of comorbid conditions. During the course of the study, noninternet participants had worse outcomes, with rates of all‐cause mortality, myocardial infarction, and stroke twice that of internet participants. Though noninternet participants completed early visits at high rates, they had significantly lower medication adherence throughout the study and completed fewer visits in the later phases of the study.
The internet is poised to become a critical tool in large‐scale, decentralized studies for reasons relating to cost, efficiency, and reach. 18 , 19 , 20 , 21 Cardiovascular clinical trials have an established history of being poorly representative of disease populations, with consistent underrepresentation of women, older adults, and ethnic minority groups. 22 , 23 , 24 , 25 , 26 Reasons for this inequity may include systemic health disparities, patient concerns, cultural values and beliefs, patient access to clinical trials or general clinical care, investigator biases affecting recruitment, and lack of physician awareness about different risks in different populations. 27 Ideally, study populations should be representative of the overall population with a given condition in order to ensure study validity and equity in the discovery of care advancements. Although internet participation may help with bringing in patients who live far from clinical centers (in contrast to the majority of our noninternet participants who were recruited face to face in a clinical setting), relying entirely on internet follow‐up may have unintended consequences such as less diversity and less representativeness. Internet access and digital literacy rates are known to be lower in older adults, Black patients, and Hispanic patients. 12 , 14 In addition, underresourced areas tend to be where digital literacy rates are lower. 13 Taken together, this forecasts the populations that will be underrepresented in internet participation‐based studies.
Participants who chose the noninternet option of ADAPTABLE follow‐up tended to have demographic characteristics associated with a less favorable profile of adverse social determinants of health. More adverse social determinants of health are associated with a higher comorbidity burden, illuminating possible reasons for the differences in baseline comorbidities between the internet and noninternet groups. 28 The greater baseline comorbidity burden in the noninternet group likely contributes to the higher observed end point rate. It is critical to recruit and retain such patients in clinical trials though, as their exclusion would have negatively affected both representativeness and statistical power. Future studies should strive to offer noninternet options of participation, as well as to introduce other ways to engage more diverse or sicker patients who are crucial to study success and the generalizability of findings.
Internet and noninternet participants also had key differences in study conduct metrics. Even after controlling for demographic/clinical characteristics and medication history, noninternet participants were more likely to switch aspirin dose and discontinue aspirin altogether. Given the open‐label nature of ADAPTABLE, multiple reasons for nonadherence were possible, such as patient preference, clinician practices, and the development of concurrent illnesses. 16 As reported in the primary paper, participants assigned to the 325 mg dose of aspirin were more likely to be nonadherent. Interestingly, our analysis showed that there were differences based on method of participation that influenced medication adherence in addition to the effect of assigned dose, even after adjusting for potential confounders associated with choosing internet or noninternet participation.
Participant retention is a significant challenge in clinical trials. One of the core advantages attributed to clinical trials using technology and newer methods is convenience for participants and removal or reduction of barriers (eg, travel costs and time loss) to continued study engagement. 19 , 29 , 30 Digital technology can also directly address retention issues in underrepresented groups, for example enabling participation of older individuals who require assistance traveling to physical study sites. Outside of enabling participation, internet‐based tools can also assist with communication and education to address mistrust and fear of experimentation in Black Americans. 18
We found internet follow‐up to be effective for participant retention in ADAPTABLE, as internet participation facilitated collection of patient‐reported outcomes and remained relatively stable throughout the late stages of the trial. ADAPTABLE also made robust efforts to engage participants through blog posts, newsletters (email and mail), and social media posts. These communication tools were helpful for patient retention and should continue to be used in future trials. A potential area of improvement would be providing training and support for internet participants, particularly for the subgroup that switched to noninternet participation and had the lowest overall visit completion rates of all participants. Because patients vary in their levels of comfort and access to technology, it can be helpful for decentralized trials to mitigate these concerns up front, in addition to offering alternative methods of participation. 20 For example, there has been prior success in providing devices to participants without internet participation as a way to allow participation in that arm. 31
Important limitations of this analysis include unbalanced participant numbers in the 2 subgroups, with many fewer patients enrolled as noninternet participants due to call center volume limitations. The noninternet group became especially small in later stages of the study, as participation dropped in both groups due to loss to follow‐up and mortality. Another source of attrition was internet participants who stopped engaging with internet follow‐up, which prompted rescue calls to reengage participants. In addition, is important to note that internet or noninternet participation was nonrandomized, so we do not know benefits internet participation itself has on outcomes and adherence outside, because the participation method chosen was influenced by underlying factors that influence internet availability and access for a given patient. Finally, we had to base our reasons for why patients preferred noninternet participation on conjecture, as we did not ascertain reasons why patients chose a participation method at enrollment.
Conclusions
In a large‐scale, decentralized study, internet methods provide ease and convenience in many regards, and savings in study administration costs. Although the internet is a supportive conduit for real‐world pragmatism and can help cast a wide net to capture patients, it has its shortcomings when it comes to creating more diverse, representative study populations. Studies limited to internet‐only methods of participation likely introduce selection bias, as we demonstrated through key differences between ADAPTABLE's internet and noninternet participant groups. Our analysis suggests that noninternet participation may promote recruitment of a diverse population, but more work is needed to understand the impact on study retention and to develop appropriate additional supports to foster ongoing engagement with study protocols. Broad inclusivity will be critical for future studies in overcoming traditional enrollment biases, particularly as underlying baseline differences appeared to drive event rates in noninternet participants.
Sources of Funding
This work was Patient‐Centered Outcomes Research Institute Award funded (ASP‐1502‐27029).
Disclosures
Dr Robertson reports currently owning stock in Dassault Systèmes (Medidata parent company). Dr Effron owns equity in and receives a pension from Eli Lilly and Company. Dr Benziger has research grants from the Agency for Healthcare Research and Quality, Patient‐Centered Outcomes Research Institute, and the US Department of Defense. She is also a site principal investigator for Amgen and Novartis trials, not related to this study. Dr Rothman has research grants from the National Institutes of Health, Agency for Healthcare Research and Quality, Patient‐Centered Outcomes Research Institute, and his family owns equity in Moderna, not related to this study. Dr Harrington has research grants and contracts from Baim Institute; Patient‐Centered Outcomes Research Institute; National Heart, Lung, and Blood Institute; Commonwealth Serum Laboratories; Janssen; is consulting with Atropos Health, Bitterroot, Bridge Bio, BMS, Element Science, serves on the Board of Directors for the American Heart Association (unpaid) and Cytokinetics. Dr Jones has research grants from Agency for Healthcare Research and Quality, AstraZeneca, American Heart Association, Bristol‐Myers Squibb, Doris Duke Charitable Foundation, Patient‐Centered Outcomes Research Institute, and has honorarium/other from American College of Radiology and Daiichi Sankyo. The remaining authors have no disclosures to report.
Supporting information
Tables S1–S2
Figure S1
This article was sent to Kwok Leung Ong, PhD, FAHA, Guest Editor, for review by expert referees, editorial decision, and final disposition.
Supplemental Material is available at https://www.ahajournals.org/doi/suppl/10.1161/JAHA.122.027899
For Sources of Funding and Disclosures, see page 11.
REFERENCES
- 1. Paul J, Seib R, Prescott T. The internet and clinical trials: background, online resources, examples and issues. J Med Internet Res. 2005;7:e5. doi: 10.2196/jmir.7.1.e5 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Litchfield J, Freeman J, Schou H, Elsley M, Fuller R, Chubb B. Is the future for clinical trials internet‐based? A cluster randomized clinical trial. Clin Trials. 2005;2:72–79. doi: 10.1191/1740774505cn069oa [DOI] [PubMed] [Google Scholar]
- 3. Durkalski V, Zhao W, Dillon C, Kim J. A web‐based clinical trial management system for a sham‐controlled multicenter clinical trial in depression. Clin Trials. 2010;7:174–182. doi: 10.1177/1740774509358748 [DOI] [PubMed] [Google Scholar]
- 4. McAlindon T, Formica M, Kabbara K, LaValley M, Lehmer M. Conducting clinical trials over the internet: feasibility study. BMJ. 2003;327:484–487. doi: 10.1136/bmj.327.7413.484 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Eagleson R, Altamirano‐Diaz L, McInnis A, Welisch E, De Jesus S, Prapavessis H, Rombeek M, Seabrook JA, Park T, Norozi K. Implementation of clinical research trials using web‐based and mobile devices: challenges and solutions. BMC Med Res Methodol. 2017;17:43. doi: 10.1186/s12874-017-0324-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Ali Z, Anderson K, Chiriac A, Andersen AD, Isberg AP, Moreno FG, Eiken A, Thomsen SF, Zibert JR. High adherence and low dropout rate in a virtual clinical study of atopic dermatitis through weekly reward‐based personalized genetic lifestyle reports. PLoS One. 2020;15:e0235500. doi: 10.1371/journal.pone.0235500 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Geyer J, Myers K, Vander Stoep A, McCarty C, Palmer N, DeSalvo A. Implementing a low‐cost web‐based clinical trial management system for community studies: a case study. Clin Trials. 2011;8:634–644. doi: 10.1177/1740774511416384 [DOI] [PubMed] [Google Scholar]
- 8. Pullen MF, Pastick KA, Williams DA, Nascene AA, Bangdiwala AS, Okafor EC, Hullsiek KH, Skipper CP, Lofgren SM, Engen N, et al. Lessons learned from conducting internet‐based randomized clinical trials during a global pandemic. Open Forum Infect Dis. 2020;8:ofaa602. doi: 10.1093/ofid/ofaa602 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Patsopoulos NA. A pragmatic view on pragmatic trials. Dialogues Clin Neurosci. 2011;13:217–224. doi: 10.31887/DCNS.2011.13.2/npatsopoulos [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Ford I, Norrie J. Pragmatic trials. N Engl J Med. 2016;375:454–463. doi: 10.1056/NEJMra1510059 [DOI] [PubMed] [Google Scholar]
- 11. Loudon K, Treweek S, Sullivan F, Donnan P, Thorpe KE, Zwarenstein M. The PRECIS‐2 tool: designing trials that are fit for purpose. BMJ. 2015;350:h2147. doi: 10.1136/bmj.h2147 [DOI] [PubMed] [Google Scholar]
- 12. Amin K, Rae M, Ramirez G, Cox C. How might internet connectivity affect health care access? Peterson‐KFF Health System Tracker. December 14, 2020. Accessed May 15, 2023. https://www.healthsystemtracker.org/chart‐collection/how‐might‐internet‐connectivity‐affect‐health‐care‐access/
- 13. Mamedova S, Pawlowski E; Department of Education . A description of U.S. adults who are not digitally literate. National Center for Education Statistics. May 2018. Accessed May 15, 2023. https://nces.ed.gov/pubs2018/2018161.pdf
- 14. Perrin A, Atske S. 7% of Americans don't use the internet. Who are they? Pew Research Center. Apr 2, 2021. Accessed May 15, 2023. https://www.pewresearch.org/short‐reads/2021/04/02/7‐of‐americans‐dont‐use‐the‐internet‐who‐are‐they/
- 15. Kakulla B. Older adults embrace tech for entertainment and day‐to‐day living. Tech trends and the 50‐plus: top 10 biggest trends. AARP Research. Dec 2021. Accessed May 15, 2023. https://www.aarp.org/research/topics/technology/info‐2022/2022‐technology‐trends‐older‐americans.html
- 16. Jones WS, Mulder H, Wruck LM, Pencina MJ, Kripalani S, Muñoz D, Crenshaw DL, Effron MB, Re RN, Gupta K, et al. Comparative effectiveness of aspirin dosing in cardiovascular disease. N Engl J Med. 2021;384:1981–1990. doi: 10.1056/NEJMoa2102137 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. Marquis‐Gravel G, Roe MT, Robertson HR, Harrington RA, Pencina MJ, Berdan LG, Hammill BG, Faulkner M, Muñoz D, Fonarow GC, et al. Rationale and design of the Aspirin Dosing‐A Patient‐centric Trial Assessing Benefits and Long‐term Effectiveness (ADAPTABLE) trial. JAMA Cardiol. 2020;5:598–607. doi: 10.1001/jamacardio.2020.0116 [DOI] [PubMed] [Google Scholar]
- 18. Inan OT, Tenaerts P, Prindiville SA, Reynolds HR, Dizon DS, Cooper‐Arnold K, Turakhia M, Pletcher MJ, Preston KL, Krumholz HM, et al. Digitizing clinical trials. NPJ Digit Med. 2020;3:101. doi: 10.1038/s41746-020-0302-y [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Norman GAV. Decentralized clinical trials. JACC: Basic Transl Sci. 2021;6:384–387. doi: 10.1016/j.jacbts.2021.01.011 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20. Agrawal GM, Moss R, Raschke R, Wurzer S, Xue J. No place like home? Stepping up the decentralization of clinical trials. RamaOnHealthcare. Jun 10, 2021. Accessed May 15, 2023. https://ramaonhealthcare.com/no‐place‐like‐home‐stepping‐up‐the‐decentralization‐of‐clinical‐trials/
- 21. Tenaerts P. Financial modeling from Tufts Center for the Study of Drug Development demonstrates substantial net benefits to sponsors who use decentralized clinical trials (DCTs) technology. Medable Matters. 2022. Accessed May 15, 2023. https://assets.website‐files.com/606a092689ad7f9fc5b493ea/61dd3e2236b2436b517a6d5d_14939357144215275752112‐tuftswhitepaper_v5.pdf
- 22. Clark LT, Watkins L, Piña IL, Elmer M, Akinboboye O, Gorham M, Jamerson B, McCullough C, Pierre C, Polis AB, et al. Increasing diversity in clinical trials: overcoming critical barriers. Curr Probl Cardiol. 2019;44:148–172. doi: 10.1016/j.cpcardiol.2018.11.002 [DOI] [PubMed] [Google Scholar]
- 23. Khan SU, Michos ED. Women in stroke trials—a tale of perpetual inequity in cardiovascular research. JAMA Neurol. 2021;78:654–656. doi: 10.1001/jamaneurol.2021.0624 [DOI] [PubMed] [Google Scholar]
- 24. Michos ED, Van Spall HGC. Increasing representation and diversity in cardiovascular clinical trial populations. Nat Rev Cardiol. 2021;18:537–538. doi: 10.1038/s41569-021-00583-8 [DOI] [PubMed] [Google Scholar]
- 25. Vitale C, Fini M, Spoletini I, Lainscak M, Seferovic P, Rosano GM. Under‐representation of elderly and women in clinical trials. Int J Cardiol. 2017;232:216–221. doi: 10.1016/j.ijcard.2017.01.018 [DOI] [PubMed] [Google Scholar]
- 26. Zhang T, Tsang W, Wijeysundera HC, Ko DT. Reporting and representation of ethnic minorities in cardiovascular trials: a systematic review. Am Heart J. 2013;166:52–57. doi: 10.1016/j.ahj.2013.03.022 [DOI] [PubMed] [Google Scholar]
- 27. Let's change history! Encourage clinical trial participation. American College of Cardiology. Jun 21, 2017. Accessed May 15, 2023. https://www.acc.org/latest‐in‐cardiology/articles/2017/05/31/17/42/lets‐change‐history‐encourage‐clinical‐trial‐participation
- 28. Jilani MH, Javed Z, Yahya T, Valero‐Elizondo J, Khan SU, Kash B, Blankstein R, Virani SS, Blaha MJ, Dubey P, et al. Social determinants of health and cardiovascular disease: current state and future directions towards healthcare equity. Curr Atheroscler Rep. 2021;23:55. doi: 10.1007/s11883-021-00949-w [DOI] [PubMed] [Google Scholar]
- 29. Benda NC, Veinot TC, Sieck CJ, Ancker JS. Broadband internet access is a social determinant of health! Am J Public Health. 2020;110:1123–1125. doi: 10.2105/ajph.2020.305784 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30. Apostolaros M, Babaian D, Corneli A, Forrest A, Hamre G, Hewett J, Podolsky L, Popat V, Randall P. Legal, regulatory, and practical issues to consider when adopting decentralized clinical trials: recommendations from the clinical trials transformation initiative. Ther Innov Regul Sci. 2020;54:779–787. doi: 10.1007/s43441-019-00006-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31. McDermott MM, Guralnik JM, Criqui MH, Ferrucci L, Zhao L, Liu K, Domanchuk K, Spring B, Tian L, Kibbe M, et al. Home‐based walking exercise in peripheral artery disease: 12‐month follow‐up of the GOALS randomized trial. J Am Heart Assoc. 2014;3:e000711. doi: 10.1161/jaha.113.000711 [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Tables S1–S2
Figure S1