Comparison of remote to hybrid methods on recruitment and retention rates in a smoking cessation trial

Abstract

Objectives:

Due to the COVD-19 pandemic, many ongoing clinical trials had to rapidly shift to using remote trials, including our smoking cessation trial within a criminal legal population. The objective of this study was to compare recruitment rate, study adherence, retention, nicotine replacement therapy adherence, and quit attempts between participants who completed the study as planned (In-Person; N = 236), after implementation of a voucher system and additional check-in appointments (Incentivized; N = 126), and after the pandemic began (Hybrid; N = 153).

Methods:

515 participants were recruited with criminal legal involvement from Birmingham, AL, and randomized to an In Vivo nicotine replacement therapy (NRT) sampling group or a standard smoking cessation counseling group, with both groups receiving 12 weeks of NRT.

Results:

There were no significant differences in any of the study outcomes between the methodology groups, suggesting that hybrid methods of research do not result in a slower recruitment pace, less visits attended, or a higher likelihood of drop-out. Completing the study remotely did not appear to impact study outcomes such as likelihood of making a quit attempt or using NRT.

Conclusions:

This study suggests that remote methods may be as effective as in-person methods in a clinical trial, although a randomized trial of these methods is needed.

1. Introduction

Smoking is the greatest cause of preventable disease, disability, and death in the U.S. and many other countries around the world (Current Cigarette Smoking Among Adults in the United States, 2023). About 11.5 % of the U.S. population smokes cigarettes (Current Cigarette Smoking Among Adults in the United States, 2023) and in certain populations (e.g., people with low income, mental illness, substance use disorders, or criminal legal involvement), prevalence rates range from 60 % to 80 % (Andrade and Kinner, 2017; Ahalt et al., 2019; Winkelman et al., 2019). Despite jails and prisons implementing smoking bans, (Kauffman et al., 2008) there is a high rate of formerly incarcerated individuals re-engaging in smoking post-release (Andrade and Kinner, 2017; Lincoln et al., 2009) as most people with criminal legal (CL) involvement remain under supervision in the community (e.g., probation, parole, specialty courts) without limitations placed on smoking behaviors.

Previous smoking cessation intervention research in individuals with CL involvement has demonstrated effective use of nicotine replacement therapy (NRT; e.g., patches, lozenges, gum) (Andrade and Kinner, 2017; Cropsey et al., 2008, 2017a). NRT is sold over-the-counter and is generally widely available and accessible (Carpenter et al., 2013). Despite this, most people prematurely discontinue NRT and use less product than is recommended, (Mersha et al., 2020a; Raupach et al., 2014) likely due to lack of knowledge and misperceptions about NRT, leading to low rates of medication adherence (Carpenter et al., 2011). Nonadherence to medications can be attributed to many reasons including high cost of medication, lack of care coordination, and individual factors (Cutler and Everett, 2010). However, medication adherence is crucial as studies with NRT (Raupach et al., 2014; Shiffman et al., 2008) and other smoking cessation medications (bupropion and varenicline (Catz et al., 2011); Cropsey et al., 2017b) have demonstrated high adherence to be associated with better smoking cessation outcomes in community and CL populations (Cropsey et al., 2017b; Mersha et al., 2020b).

1.1. Remote clinical trials research methods

Slow recruitment pace, low enrollment, and high attrition are some of the main challenges in conducting clinical trials (Mahoney et al., 2021; Nipp et al., 2019; Rodríguez-Torres et al., 2021). However, remote trials provide the opportunity to reach participants who might not be able to participate otherwise due to time commitments, lack of transportation, or other reasons (Mahoney et al., 2021; Nipp et al., 2019; Rodríguez-Torres et al., 2021; Alemayehu et al., 2021). In tobacco research, many researchers have incorporated remote methods into study design. Methods such as text-messaging, delivery of therapy or assessments pushed to smartphones as well as internet-based smoking cessation interventions have all recently been developed (Taylor et al., 2017; Whittaker et al., 2019) and smartphone-enabled carbon monoxide (CO) monitors (Tuck et al., 2021) or mailed in (cotinine) saliva samples (Thrul et al., 2023) can be used to receive remote biochemical verification of smoking status. Studies directly comparing in-person to remote completion have found similar rates of participation (e.g., number of visits completed, saliva samples given, rates of cessation) (Mahoney et al., 2021; Carlson et al., 2012). Only one pilot trial has examined remote methods in the CL population and demonstrated benefit to persons in rural prisons (Valera et al., 2021). Overall, while these previous studies show promise for remote methods, they were also generally conducted with relatively small samples, limiting opportunities to examine differences between in person and remote study procedures.

1.2. The current study

Given the preliminary success of giving free samples of NRT to people who smoke to help them quit in the general population (Jardin et al., 2014) and based on the pilot interventions among people who smoke who are low-income (Cropsey et al., 2021) and in the CL population (Cropsey et al., 2017a) this study provided an intervention that was intended to increase medication adherence as a way to facilitate successful cessation attempts. We compared study outcomes between participants who 1) completed the study as planned (In-Person), 2) were recruited after incentives and check-in appointments were implemented (Incentivized), and 3) were recruited after the pandemic began (Hybrid). The Hybrid group consists of participants who were incentivized but completed all study procedures remotely except for the baseline visit. Main outcomes of the study are detailed in another publication (Cropsey et al., 2024).

2. Method

2.1. Participants

Participants (N_l=515) were recruited from flyers posted in the UAB Substance Use Treatment programs serving individuals under CL supervision. Inclusion criteria were: (a) under community CL supervision over the next 6 months, (b) smoking at least 5 cigarettes/day for the past year (Higgins et al., 2020; Carpenter et al., 2021) (c) age 18 years or older, (d) able to read and speak English, (e) provided contact information for at least two people if unable to contact participant (f) living in an environment that allows smoking, (g) access to a smartphone or personal email address.

Additionally, participants could not (a) be pregnant or breastfeeding, (b) have cognitive impairment or untreated mental illness that interferes with informed consent, (c) have experienced (within 6 months) post-myocardial infarction or untreated severe angina, (d) have a known sensitivity to NRT or adhesive products (e) exclusively use other tobacco products (e.g., cigars, e-cigarettes; although concurrent use of other tobacco products was not an exclusion criterion), or (f) be currently receiving treatment to quit smoking.

Recruitment for this clinical trial progressed in three phases. First, flyers were posted at local CL system buildings and substance use disorder treatment sites. Participants who were recruited with this method completed all intervention and data collection appointments in person as originally planned, and this group is referred to as the In-Person group (n = 236). With the aim of increasing the recruitment rate, snowball recruitment was implemented, and existing participants were financially incentivized to refer new participants. The group of participants recruited after implementing these incentives and before the start of the COVID-19 pandemic are referred to as the Incentivized group (n = 126). All Incentivized participants completed the baseline procedures in person. All of their other appointments were completed in person if they occurred before the start of the COVID-19 pandemic, but participants recruited near the end of the Incentivized phase completed some of their appointments remotely. Because of the COVID-19 pandemic, recruitment was paused on March 13, 2020, and restarted on June 25, 2020, with Hybrid participants (n = 153), who completed baseline in person and all other appointments remotely.

2.2. Procedures

The study was approved by the university’s institutional review board and all participants provided informed consent. At the in-person baseline session, smoking status was verified with breath CO using the Vitalograph CO monitor and urine cotinine. After confirming eligibility, participants completed baseline assessment measures and were randomized to one of two conditions (1:1). The randomization scheme was independent of participants’ status as In-Person, Incentivized, or Hybrid. The intervention (In Vivo) group sampled NRT at each weekly session (Session 1: Patch, Session 2: Lozenge, Session 3: Combined Patch & Lozenge) and received the sampled product to use between appointments. Participants were asked about expectations and experience with medication during the sessions. The control group received behavioral smoking cessation counseling during the first three sessions such as cognitive and behavioral strategies for coping with cravings and withdrawal, stimulus control, and relaxation techniques. After the third session, the control group also received combo NRT; however, NRT was not used In Vivo during counseling sessions and was dispensed like what they would receive from a pharmacy. The final session differed for both groups with the In Vivo group focusing on participants’ experiences with NRT the prior week, while the control group focused on gains made during the intervention and relapse prevention. Both the In Vivo and control groups received combo NRT to use from the third counseling session through to 12 weeks after the final counseling session. This amount of NRT was provided to participants to match recommended durations of post-quit date NRT use (Hartmann-Boyce et al., 2018).

Participants completed five subsequent visits (week 8 & 12 of intervention, months 1, 3, and 6 follow-up) during which participants completed questionnaires and exhaled CO was collected as an indicator of smoking status. Initially CO was measured at each visit using the Vitalograph CO monitor; however, when recruitment resumed following the COVID lockdown, participants were taught how to use an iCO Smokerlyzer device at the baseline appointment to use remotely for each subsequent visit. Given the different measurement values between the Vitalograph and iCO, (Tonkin et al., 2023) participants provided exhaled CO at baseline with both devices to allow comparison with the exclusive Vitalograph group. Participants who completed the entire study received $440 compensation.

Participants were given multiple reminders when they had upcoming study appointments. To assess study retention and adherence for all groups, a time point was considered missing if the participant’s CO reading or weekly smoking behavior survey was incomplete. Missing information on individual items was minimal given the nature of REDCap questionnaires, which prevents skipping items.

2.3. Measures

Data was collected as part of the parent study for all outcomes. The measures below were used in this secondary analysis.

2.3.1. Demographic form

Demographic information such as age, sex, number of children, employment, and education level were collected at their baseline appointment. The continuous variable for number of children in the house was recoded into the following categories: 0, 1, 2, or 3 or more children. Employment status over the past three years was recoded from the original eight categories: full time, part time (regular hours), part time (irregular hours), student, military service, retired/disability, unemployed, or in controlled environment into: full-time, part-time, unemployed, or other by collapsing the prior categories. The variable for past 30-day work history was created by creating two groups (yes/no) based on the question “How many days were you paid for working in the past 30 days?”.

2.3.2. Perceived stress scale

This is a 10-item measure assessing one’s perceived stress over the past month. There are five answer choices ranging from “never” to “very often.” Total scores can range from 0 to 40, with a score of 27–40 indicating high perceived stress.

Weekly Smoking Behavior Survey

This measure assesses smoking behavior over the past week and collects information such as average cigarettes smoked per day, other tobacco product use, and 24-hour quit attempts made.

2.3.3. NRT use calendar

These measures collect information about the number of nicotine patches used and nicotine lozenges used every day over the past week. These measures are used to examine NRT adherence at session 4.

2.4. Data analytic approach

2.4.1. Preliminary data analysis

Descriptive and graphical analyses for all variables of interest were conducted to identify outliers and out-of-range values. Preliminary analyses verified that the assumptions specific to the statistical techniques that were used were met, and adjustments were made for heteroscedasticity and overdispersion, in linear and Poisson regression models, respectively.

2.4.2. Primary analysis

SPSS (IBM Corp., 2022) version 29 and RStudio (version 4.2.2, R Core Team, 2022) were used to conduct all analyses. Descriptive statistics were obtained for all sociodemographic and primary study variables. Recruitment rate was evaluated using Poisson regression with aggregated data, comparing the number of participants recruited for the duration in days of each methodology group (as offset variable). Methodology group, sex, employment (paid over past-30 days), and perceived stress score (split into quartiles) were used as adjusting covariates. The unadjusted Poisson model included data from 3 aggregated profiles (one for each methodology group), while the adjusted Poisson model included data from 48 aggregated profiles (resulting from the combinations of explanatory variable categories).

For all other analyses, the role of the methodology group was examined as an effect modifier with interaction terms between methodology group (e.g., In-Person, Incentivized, Hybrid) and treatment group (e.g., Intervention, Control), to determine if methodology group resulted in differential effect of the treatment. The study adherence variable was computed by taking the total number of visits a participant attended and dividing it by 10 (the total number of visits possible) to create a fraction between 0 and 1. Study adherence was evaluated using fractional logistic regression models for a continuous proportion bounded to the interval 0–1 to examine the effects of treatment group, methodology group, and the treatment by methodology group interaction. The following variables were also included as covariates: age, sex, employment (status over the past three years, paid over the last 30 days), children in the home, education, and perceived stress score. The percentage of missing CO readings from remote visits (i.e., iCO readings from the Hybrid group) was calculated for each visit and overall.

Patch adherence was calculated by computing the proportion of hours the patch was worn divided by 24 h for each day of the week and then by taking the average for the week. Similarly for lozenge adherence, the number of lozenges used for each day was divided by eight (the recommended dosage). If a participant took more than eight lozenges on any given day, the data was capped so that they could not have a higher proportion than one. An average of the seven days was then calculated to create the weekly lozenge adherence variable. Fractional logistic regression models were fitted to the NRT adherence variables with treatment group, methodology group, and the treatment by methodology group interaction as main explanatory variables. The following variables were also included as covariates: age, sex, employment (status over the past three years, paid over the last 30 days), children in the home, education, and perceived stress score.

Study retention and quit attempts at session four were evaluated using logistic regression. If a participant made any quit attempt between session three and four, they were coded as “yes,” otherwise participants that did not make any quit attempts were coded as “no.” Main explanatory variables for the logistic models were treatment group, methodology group, and the treatment by methodology group interaction. Age, sex, employment (status over the past three years, paid over the last 30 days), number of children in the home, education, and perceived stress score were included as covariates. Similarly, those that attended the 6-month follow-up visit were coded as “yes” and those that did not were coded as “no” for examining study retention using logistic regression.

3. Results

3.1. Demographic information

Descriptive statistics for sociodemographic variables for the overall sample and by methodology group (In-Person, Incentivized, and Hybrid) are reported in Table 1. Descriptive statistics are also included for primary study outcomes in Table 1.

Table 1.

Sample Characteristics.

	Overall (n = 515)		Group Differences*	In-Person (n = 236)		Incentivized (n = 126)		Hybrid (n = 153)
	M	SD	p value	M	SD	M	SD	M	SD
Age nmissing = 0	39.9	10.6	.12	39.6	11.2	41.4	10.2	39.1	10.1
Perceived Stress	17.3	7.2	< .01	18.4	7.4	17.4	6.9	15.6	7.0
% Patch Adherence at S4	70.7	34.1	.70	69.4	34.4	70.8	35.7	72.8	32.3
% Lozenge Adherence at S4	59.3	37.5	.45	58.7	38.1	56.4	36.9	62.6	37.2
% of Visits Completed	78.3	30.5	.56	77.8	31.1	76.7	30.3	77.8	31.1
	n	%		n	%	n	%	n	%
Sex nmissing = 0			< .01
Female	247	48	-	105	44.5	34	27.0	108	70.6
Male	268	52	-	131	55.5	92	73.0	45	29.4
Race nmissing = 0			< .01
White/Caucasian	289	56.1	-	112	47.5	58	46.0	119	77.8
African American/Black	213	41.4	-	121	51.3	66	52.4	26	17.0
Bi-racial	13	2.5	-	3	1.3	2	1.6	8	5.2
Ethnicity nmissing = 0			.25
Hispanic	15	2.9	-	4	1.7	4	3.2	7	4.6
Non-Hispanic	500	97.1	-	232	98.3	122	96.8	146	95.4
Randomization nmissing = 0			.98
In Vivo (Intervention)	259	50.3	-	119	45.9	63	14.3	77	29.7
Counseling (Control)	256	49.7		116	45.3	64	25.0	76	29.7
# of children in the home nmissing = 0			.31
None	115	22.3	-	59	25.0	30	23.8	26	17.0
1 child	100	19.4	-	44	18.6	29	23.0	27	17.6
2 children	119	23.1	-	54	22.9	23	18.3	42	27.5
3 or more children	181	35.1	-	79	33.5	44	34.9	58	37.9
Education nmissing = 0			.09
Less than high school	127	24.7	-	60	25.4	33	26.2	34	22.2
High school graduate/GED	200	38.8	-	92	39.0	57	45.2	51	33.3
More than high school	188	36.5	-	84	35.6	36	28.6	68	44.4
Employment last 3 years nmissing = 0			.02
Full-time	187	36.3	-	76	32.2	54	42.9	57	37.3
Part-time	94	18.3	-	38	16.1	28	22.2	28	18.3
Unemployed	120	23.3	-	56	23.7	23	18.3	41	26.8
Retired/disabled	58	11.3	-	37	15.7	13	10.3	8	5.2
Other	56	10.9	-	29	12.3	8	6.4	19	12.4
Employed/Paid in the past 30 days nmissing = 1			.01
Yes	136	26.5	-	75	31.8	22	17.5	39	25.5
No	378	73.5	-	161	68.2	103	81.8	114	74.5
Quit Attempt Made at S4	160	38.6	.13	74	38.9	44	42.7	42	34.4

Note. GED: Graduate Equivalency Degree; CPD: Cigarettes Per Day; S4: treatment session 4. Missing data values are due to participants being allowed to decline answering certain questions.

^*Significance determined by chi-square tests and one-way ANOVAs between groups.

3.2. Recruitment rate

The overall test of group differences in recruitment rate was significant (χ²(2) = 12.07, p = .002). The Incentivized group (.41 participants/day) was recruited faster than the In-Person (.29 participants/day) and Hybrid (.28 participants/day) groups. Fig. 1 depicts the cumulative recruitment for each group. After adding confounding variables that were significantly different between groups, (employment, sex, PSS score), the recruitment rate difference between the Incentivized group and In-Person group remained; however, the difference between the Hybrid group and Incentivized group was not significant (see Table 2).

Fig. 1. Cumulative Enrollment by Group.

Note. Recruitment for the In-Person group began on Feb. 8, 2017, and lasted 818 days. The Incentivized group recruitment then began on day 818 and lasted for 311 days, after which there was an 104-day pause in recruitment due to the COVID-19 pandemic. Recruitment for the Hybrid group started on day 1233 and lasted 551 days.

Table 2.

Recruitment Rate by Group.

Unadjusted Analyses †
Group	Recruitment in persons/day	Lower CI	Upper CI	Standard Error
In-Person	0.29	0.25	0.33	0.02
Incentivized	0.41	0.34	0.49	0.04
Hybrid	0.28	0.24	0.33	0.02
Rate Comparisons	Rate Ratios	Lower CI	Upper CI	Significance
Incentivized vs. In-Person	1.01	1.00	1.02	0.00
Incentivized vs. Hybrid	1.01	1.00	1.01	0.01
Hybrid vs. In-Person	1.00	1.00	1.01	0.28
Adjusted Analyses ‡
Group	Recruitment in persons/day	Lower CI	Upper CI	Standard Error
In-Person	0.02	0.01	0.02	.00
Incentivized	0.03	0.02	0.03	0.00
Hybrid	0.02	0.02	0.02	0.00
Rate Comparisons	Rate Ratios	Lower CI	Upper CI	Significance
Incentivized vs. In-Person	1.48	1.10	1.99	0.01
Incentivized vs. Hybrid	1.34	0.97	1.85	0.08
Hybrid vs. In-Person	1.11	0.84	1.46	0.47

Note. Pairwise comparisons are made using the model adjusted for covariates (PSS, sex, & employment).

^†3 aggregated profiles: on average 172 recruited participants in 560 days per profile, rate: 172/560 = 0.31.

^‡48 aggregated profiles: on average 11 recruited participants in 507 days per profile, rate: 11/507 = 0.02.

3.3. Study adherence

Type II tests for treatment group (χ²(1)= 0.19, p = .66), methodology group (χ²(2)= 0.559, p = .76), and the treatment and methodology group interaction (χ²(2)= 1.71, p = .42) indicated these variables were not significant predictors of study adherence. The overall percentage of missing CO readings from remote visits was 9.7 % (see Supplemental Table 1).

3.4. Study retention

While retention among all three groups was relatively similar during the intervention weeks (W1-W4), by M1 the Hybrid group (70.6 %) had better retention than the In-Person (67.8 %) or Incentivized (61.9 %) groups. Similarly, at month 3, the Hybrid group (67.3 %) had better retention than the In-Person (60.6 %) or Incentivized (57.9 %) groups. By month 6, the Hybrid group had a retention rate of 66 % compared to In-Person (56.4 %) or Incentivized (56.3 %). Overall, 304 participants (59 %) completed the six-month follow-up visit and one case was excluded from the analysis because of missing covariate data. The omnibus logistic regression model test was statistically significant, χ²(17) = 40.73, p = .001. However, the model correctly classified only 62.1 % of cases (Nagelkerke R²=.10). Treatment group, methodology group, and the treatment and methodology group interaction were not significant predictors.

3.5. NRT adherence

416 participants (80.8 %) completed the session four visit and six of these cases were excluded from this analysis due to missing data. While the treatment group by methodology group interaction was not significant, the main effect of intervention was significant, consistent with the main outcomes study (Cropsey et al., 2024) (Fig. 2). Participants in the intervention group had on average 31.8 % higher adherence to the recommended number of lozenges (at least eight) per day than the standard counseling group (73.4 % vs. 41.6 %, OR=3.86, p < .001). For nicotine patch adherence, a similar, though less-strong, effect of treatment group was evident. Participants in the intervention group had on average, 12.4 % higher adherence to use one patch/day for 24 h over the final week of the intervention period than participants in the standard counseling group (76.3 % vs. 63.9 %, OR= 1.82, p < .001).

Fig. 2. Nicotine Replacement Therapy Adherence Between Sessions 3 and 4.

Note. Lozenge adherence was measured as proportion of hours of use. Patch adherence was measured as proportion of days of use. Black dots indicate mean point estimates of those proportions. The bars are 95 % confidence intervals around the mean.

3.6. Quit attempts

416 participants (80.8 %) completed the session four visit and two of these cases were excluded from this analysis due to missing data. The omnibus test for the logistic regression model was not statistically significant, χ²(17) = 23.48, p = .13. The treatment by methodology group interaction was not significant; however, after removing the interaction from the model, the results suggested a main effect of treatment group (OR=1.62, 95 % CI [1.07, 2.45]). Compared to their control counterparts, participants in the In Vivo group had 62 % higher odds of reporting a quit attempt at session four.

4. Discussion

This study was one of the first to compare remote/hybrid to in-person study methods. Overall, the Hybrid group performed similarly to other groups on recruitment rate, appointment adherence, NRT adherence, and quit attempts. One possible reason that the recruitment rate for the Hybrid group was not faster than the other groups is because of the impact of COVID-19, and not because of the difference in methodology groups. Some individuals, especially people who smoke, may not have wanted to risk contracting the COVID-19 virus knowing that smoking put them at a higher risk for complications (Haddad et al., 2021). To add to that possibility, the Hybrid group reported significantly lower levels of stress at baseline than the other groups (see Table 1), which may mean that people with higher levels of stress chose not to participate in research during the pandemic. Order effects may offer another explanation, as it is typically harder to find participants for a large-scale study as time passes, and the Hybrid group was composed of the last 153 participants out of 515.

The finding that the fully remote Hybrid group had better retention for follow-up visits compared to requiring in-person appointments may indicate that participants view remote appointments as less burdensome and/or easier to schedule than coming in person. Additional research with randomization to in-person or remote methods for both intervention and control participants is needed to replicate these results, as well as confirm that remote methods do not dilute the quality of data or effectiveness of interventions. Better retention for remote research appointments, without sacrificing fidelity in procedures, would be beneficial for researchers. Remote research typically allows for a more diverse sample of participants to be recruited. Remote methods may also reduce potential barriers (e.g., travel time, transportation needs, childcare needs) and increases participation across socioeconomic groups, as 97 % of Americans with household incomes under $30,000 own a cellphone (Sharma et al., 2022). Of course, remote participants need some increased technology literacy (e.g., ability to complete online surveys, join video calls), but technological literacy is improving, especially since the COVID-19 pandemic (Martínez-Alcalá et al., 2021).

4.1. Strengths and limitations

A major limitation is that the quasi-experimental nature of the study design does not afford the opportunity to discern how much the results are affected by COVID-19 (e.g., getting sick, using technology more and in some cases using for the first time, working from home or losing a job, different childcare responsibilities). Similarly, participants were not randomized to the different approaches but were in these groups due to COVID-19; thus, a study design in which participants are randomized to in-person vs. remote groups would allow for a better understanding of recruitment rate and study protocol adherence as well as treatment outcomes due to these different approaches. Additionally, the present results are only generalizable to people with some technological ability, as people without a smartphone or email address were excluded. It is likely that people who do not own or commonly use technological devices would have lower adherence and study retention in studies with substantial online components. Finally, we took a conservative approach to include many variables in the model, controlling for potential confounds given the non-random assignment of methodology groups. It is possible that overlapping variance between these variables (i.e., multicollinearity) reduced statistical power to the point where identifying real treatment by methodology group effects was too low. In the absence of a randomized controlled trial, propensity score matching could be used to reduce the influence of confounds between groups in a more quasi-experimental design where random assignment to in-person vs. remote is not used. However, this study is one of the first to examine the impact of transitioning to hybrid methods and demonstrates that study outcomes of interest remained largely unaffected. Strengths of this study include a robust sample size in each of the three groups and incorporation of the same measures and procedures across all groups despite the onset of COVID-19.

5. Conclusions

COVID-19 accelerated the adoption of remote methods given the risks of in-person contact with research participants (Dahne et al., 2020; Izmailova et al., 2020). For many research teams, this was the first time they started obtaining digital consent, using video conferencing platforms to substitute for in-person visits, and transitioning from pen and paper assessments to online assessments (McDermott and Newman, 2021; Saberi, 2020). While this comparison was unplanned due to the pandemic’s effects on the in-person protocol, there are benefits to digitizing clinical trials that should be considered outside of the context of the pandemic, (Inan et al., 2020) such as increased reach of participants for recruitment and reduced costs. While remote clinical trials have many advantages, they also have some limitations (Mahoney et al., 2021). For example, it is more difficult to monitor participants from their own home, therefore protocol standardization is decreased, and the intervention setting is not uniform (Chiamulera et al., 2021). Even more concerning is the possibility that participants can deceitfully complete screening surveys and enroll in studies they are not eligible for or, alternatively, participate in the same study more than once, compromising the integrity of the data (Teitcher et al., 2015). More research is needed to be able to accurately weigh these costs and benefits when designing a study.

Appendix A. Supporting information

Supplementary data associated with this article can be found in the online version at doi:10.1016/j.drugalcdep.2025.112976.