Methods-Motivational Interviewing Approach for Enhanced Retention and Attendance

Abstract

Introduction:

Suboptimal and differential participant engagement in randomized trials—including retention at primary outcome assessments and attendance at intervention sessions—undermines rigor, internal validity, and trial conclusions.

Methods:

First, this study describes the Methods-Motivational Interviewing approach and strategies for implementation. This approach engages potential participants before randomization via interactive, prerequisite orientation sessions that illustrate the scientific rationale behind trial methods in accessible language and use motivational interviewing to diffuse ambivalence about participation. Then, this study examines potential improvements in retention (proportion of participants assessed at follow-up visits) and attendance (e.g., mean percentage of intervention sessions attended, percentage of participants who attended 0 sessions) in 3 randomized weight-management trials that quickly added prerequisite orientations to their protocols following early signs of suboptimal or differential participant engagement (Supporting Health by Integrating Nutrition and Exercise [SHINE, 2009–2013, n=194]; Get Social [2016–2020, n=217]; GestationaL Weight Gain and Optimal Wellness [GLOW, 2014–2018, n=389]). Using a pre–post analytical design, adjusted estimates from regression models controlling for condition and assessment timepoint (analyses from 2020) are reported.

Results:

After adding prerequisite orientations, all 3 trials attained higher participant engagement. Retention at assessments was 11.4% and 17.3% higher (Get Social, SHINE). Mean percentage attendance at intervention sessions was 8.8% higher (GLOW) and 10.1% less participants attended 0 intervention sessions (Get Social). Descriptively, all remaining retention and attendance outcomes were consistently higher, but non-significant. Across trials, adding prerequisite orientations did not affect the proportion of eligible participants enrolled or baseline demographics.

Conclusions:

The Methods-Motivational Interviewing approach shows promise for increasing rigor of randomized trials and is readily adaptable to in-person, webinar, and conference call formats.

Trial Registration:

All 3 trials are registered at www.clinicaltrials.gov (SHINE: NCT00960414; Get Social: NCT02646618; GLOW: NCT02130232).

INTRODUCTION

Participant engagement in randomized behavioral intervention trials is typically assessed as participant retention at follow-up assessments of trial primary outcomes and participant attendance at behavioral intervention classes. Yet, suboptimal and/or differential trial retention and intervention attendance can distort the validity of trial conclusions, diminish rigor, and jeopardize investments made in the trial by investigators, funders, and participants themselves.

In 2005, Goldberg and Kiernan¹ first described the innovative Methods-Motivational Interviewing (MMI) approach of prerequisite orientation sessions to increase retention in randomized trials. Prerequisite orientations—interactive and purposely held well before participants enroll—go beyond providing trial information. Rather, prerequisite orientations meaningfully involve participants in actively learning about the trial design and research question at hand, explain the scientific rationale behind trial methods, diffuse ambivalence about the pros/cons that participants generate when considering participation, and explicitly encourage potential participants to consider 2 commitments including to self (by changing target behaviors) and to the trial methods (by completing all trial assessments independently of personal success).

The MMI approach, including the interactive prerequisite orientations, was descriptively associated with high retention in 2 initial trials.¹,² The MMI approach and related strategies have since been integrated in numerous randomized behavioral intervention trials across a wide variety of health problems and populations including e-health programs for adults with unhealthy lifestyles,³,⁴ family-based programs promoting healthy lifestyles,⁵ self-management of type 1 diabetes for adolescents,⁶ healthy diets for asthma control among adults with uncontrolled asthma,⁷ faith-based programs,⁸ exercise programs for diverse and vulnerable populations such as sedentary Latino adults and family caregivers,⁹–¹¹ digital cognitive behavioral therapy to reduce insomnia symptoms among pregnant women,¹² in-person cognitive behavioral therapy for depression among patients with chronic pain,¹³ and others.¹⁴

The current study first describes MMI’s core constructs and key strategies. Then, using a pre–post analytic design, the study examines whether participant retention and attendance increased in 3 randomized weight management intervention trials that quickly added prerequisite orientations to their protocols in response to early signs of suboptimal or differential trial retention and intervention attendance. Finally, the study describes each trial’s innovative adaptations to the MMI approach.

METHODS

MMI Approach

Informed by community-based participatory research principles in which participants are considered partners,¹⁵ the MMI approach leverages interactive, prerequisite orientation sessions, held well before randomization, informed by 4 core constructs: set clear participant expectations (e.g., be transparent about lengthy assessments), explain the scientific rationale behind trial methods in easy-to-understand language (e.g., why randomize and impact of poor retention on trial conclusions), diffuse ambivalence about research participation using motivational interviewing (e.g., generate pros/cons of trial participation), and make 2 commitments explicit (to self and to trial methods). Ideally, the MMI approach should be implemented before a trial starts. Table 1 presents MMI’s core constructs and key strategies, as well as specific examples of the originally implemented procedures and adapted procedures in the 3 trials (e.g., online webinar/webchat, conference calls).

Table 1.

Methods-Motivational Interviewing Approach to Prerequisite Orientation Sessions: Core Constructs, Key Strategies, Originally Implemented Procedures, and Adapted Procedures

Core constructs	Key strategies	Originally implemented proceduresa	Adapted procedures (corresponding trial)
Set clear expectations for participants (e.g., trial staff are transparent about lengthy assessments)	Ensure participants have the time and space to consider the full commitment of participation with full information about the trial	Describe the research question and why it is compelling and important relative to prior trials; inspire excitement about the research question Explain the rationale for assessments Provide participants with a handout of required activities in a folder during the orientation session to take home Include a copy of an unsigned trial informed consent form in their folder to read ahead of possible enrollment	Participants, not trial staff, are required to initiate the next steps of the enrollment process after the prerequisite orientation session (SHINE) Trial information sent to participants prior to the prerequisite orientation session (GLOW)
Explain the scientific principles behind trial methods (e.g., randomization and retention)	Consider and interact with participants as partners in the research process Manage (do not ignore) participants’ expectations and preferences for a certain trial condition by illustrating the scientific purpose of randomization Create a non-judgmental space for participants to return to follow-up assessments, regardless of personal progress during the trial (e.g., lack of weight loss success)	Provide an easy-to-understand “Research 101” educational module and handout explaining the scientific rationale for trial methods (e.g., retention), foster research literacy for this trial (or future trials) Review an infographic illustrating scientific value of a “true picture”b versus biased trial results if only some participants are assessed at follow-up Use multiple delivery channels (e.g., visually powerful trial timeline); avoid academic jargon	Discussed the importance of retention at trial assessments, regardless of personal progress during the trial (GLOW)
Diffuse ambivalence about participating in research using motivational interviewing (e.g., trial staff remain neutral regarding participation)	Explore participants’ willingness to be randomized to either trial condition Explicitly discuss the difficulties of making behavior changes and participating in research trials Manage expectations about the trial outcomes for individuals Share ideas verbally with the group to recognize and normalize feelings of ambivalence that may be ignored during the informed consent process	Within a large group orientation, ask breakout groups of 3–4 people to generate 2 pros and 2 cons for the most difficult comparison faced during enrollment (e.g., being in the research trial or not in the trial) Ask breakout groups to share pros and cons with larger group; summarize responses verbally and on white board; begin with (1) cons of not being in the trial, (2) pros of not being in the trial, (3) pros of being in the trial, and (4) cons of being in the triala Avoid taking a “pro-change” position Differentiate between a research-based intervention and similar programs offered in the community	Facilitated online webchat during webinar where participants shared their thoughts about the pros/cons of the 2 trial conditions (e.g., standard intervention and new intervention) in the chat box, and the moderator discussed all responses aloud to whole group; followed original order for discussion except ended with pros of being in the new intervention condition (Get Social) Facilitated the pros/cons discussion within the large group (i.e., no breakout groups); no specific order followed for the discussion (SHINE) Facilitated the pros/cons discussion with individuals or small groups at once; no specific order followed for the discussion (GLOW)
Make 2 commitments explicit (to self and to trial methods)	Enhance participant understanding of the importance of excellent trial retention, regardless of individual experience If participants choose to enroll, 2 commitments discussed at the prerequisite orientation session can be built on and revisited later by staff (e.g., if follow-up is challenging)	Facilitate the discussion during the orientation about the difference between and importance of the 2 commitments Acknowledge possible negative feelings about long assessments and rationale for similar measures across follow-up assessments (that may feel repetitive)	None

^aGoldberg JH, Kiernan M. Innovative techniques to address retention in a behavioral weight-loss trial. Health Educ Res 2005;20(4):439–447

^bKiernan M, Oppezzo MA, Resnicow K, Alexander GL. Effects of a methodological infographic on research participants’ knowledge, transparency, and trust. Health Psychol. 2018;37(8):782–786.

SHINE, Supporting Health by Integrating Nutrition and Exercise; GLOW, GestationaL Weight Gain and Optimal Wellness.

Trial Characteristics

This analysis examined the impact of adding prerequisite orientations on retention and attendance across three randomized trials, each of which compared a multisession behavioral weight management program for adults with a control condition (Table 2). These trials were a convenience sample, selected because the MMI approach was implemented in ongoing trials (not before the start).

Table 2.

Characteristics of Trials and Prerequisite Orientation Sessions

Characteristics	SHINE	Get Social	GLOW
Trial characteristics
Sample size for present analysis	N=194	N=217	N=389
Dates of data collection Population, location	2009–2013	2016–2020	2014–2018
	Adults with obesity (BMI 30–45 kg/m2) and elevated waist circumference (men, >102 cm; women, >88 cm); California	Adults with overweight or obesity (BMI 27–45 kg/m2); Massachusetts	Pregnant women with overweight or obesity (BMI 25–40 kg/m2); California
Topic	Efficacy trial of adding mindfulness training to a weight loss program	Non-inferiority trial of an online social network-delivered versus clinic-delivered weight loss program	Efficacy trial of a lifestyle program to prevent excess gestational weight gain
Trial design	2 groups, randomized	2 groups, randomized	2 groups, randomized
Trial conditions
Intervention condition	Mindfulness	Get social	Getting in balance
Intervention description	Group sessions (16) with mindfulness content, delivered in person: 12 weekly, 3 bimonthly, and 1 monthly sessiosns over 5.5 months, including an all-day retreat	Group modules (23a) delivered online via private Twitter group: 16 weekly, 4 biweekly, and 6 monthly sessions over 12 months	Individual sessions (13) delivered weekly via telephone (11 sessions) and in person (2 sessions) over ≈3.5 months during pregnancy
Control condition	Active control	Traditional	Usual care
Control description	Group sessions (16) with no mindfulness content, delivered in person: 12 weekly, 3 bimonthly, and 1 monthly sessions over 5.5 months, including an all-day retreat	Group sessions (22) delivered in person: 16 weekly, 4 biweekly, and 6 monthly sessions over 12 months	Usual care (no sessions)
Assessment visits	3, 6, 9 and 18 months	6 and 12 months	32 weeks gestation, 6 and 12 months postpartum
Orientation session characteristics
Format	In person, with PowerPoint slides	Via webinar, with PowerPoint slides	By telephone conference calls, with PowerPoint slides sent in advance
Facilitator	Study project director	Postdoctoral fellow or research assistant	Study co-investigator or project manager
Typical group size	8–20 participants	8–15 participants	1–3 participants
Length	90 minutes	60 minutes	45 minutes

^aSessions 22 and 23 collapsed for comparability in analyses across conditions.

SHINE, Supporting Health by Integrating Nutrition and Exercise; GLOW, GestationaL Weight Gain and Optimal Wellness.

The first trial, Supporting Health by Integrating Nutrition and Exercise (SHINE) was an efficacy trial testing the addition of mindfulness training to a diet–exercise weight loss program (data collected in 2009–2013, n=194).¹⁶ The 16-session mindfulness-based training weight loss program (Mindfulness) was compared against an active control program of behavioral weight loss with identical diet–exercise guidelines (Active Control), with follow-up assessments at 6 and 18 months post-randomization.

The second trial, Get Social, was a non-inferiority trial of an online social network–delivered weight loss program (data collected in 2016–2020, n=217).¹⁷ The 23-module Twitter-based program (Get Social) was compared against a 22-session in-person, group-based program (Traditional), with follow-up assessments at 6 and 12 months post-randomization. Get Social Modules 22 and 23 were collapsed for comparability across conditions in analyses. The sample in the present analysis included participants randomized in the first 6 waves while the study was administered at the University of Massachusetts Medical School, before it was moved to the University of Connecticut. The 3 final waves of participants were excluded to reduce potential confounding by participant characteristics and other factors influencing participant engagement at a new institution.

The third trial, GestationaL Weight Gain and Optimal Wellness (GLOW) was an efficacy trial of a 13-session lifestyle program primarily delivered by telephone to prevent excess gestational weight gain (Getting in Balance), which was compared with usual medical care (data collected in 2014–2018, n=389).¹⁸,¹⁹ Pregnant women were randomized into the trial at 8–15 weeks gestation, with follow-up assessments at 32 weeks gestation, 6 months postpartum, and 12 months postpartum. Here, the GLOW sample excluded 9 women who experienced pregnancy loss before the first follow-up assessment, and thus were ineligible for remaining assessments.

All 3 trials were approved by IRBs and obtained participants’ informed consent in accordance with their institution’s ethical standards.

Implementation of MMI Approach

Each trial moved to implement the MMI approach of prerequisite orientations in response to early concerns about suboptimal or differential participant engagement, and adapted the approach for their unique context (Table 1). Participants here are defined as either before (enrolled before prerequisite orientations were added) or after (enrolled after the prerequisite orientations were added), and by definition cannot be both. SHINE initiated in-person, group-based prerequisite orientations in response to suboptimal and differential retention at follow-up assessments (43.8% [n=85/194] of participants enrolled before prerequisite orientations were added). Get Social initiated interactive group webinar-based prerequisite orientations in response to suboptimal and differential attendance, including early indications that some participants did not attend any of the in-person, group-based intervention sessions (Traditional) (34.6% [n=75/217] of University of Massachusetts Medical School participants before prerequisite orientations were added). Although GLOW retention at follow-up assessments initially appeared high overall, GLOW initiated orientations via telephone conference calls in response to small differences in retention between trial conditions and to enhance attendance (41.1% [n=160/389] of participants before prerequisite orientations were added).

Measures

Trial retention at assessments is presented by trial condition for each primary outcome assessment (e.g., 6-month and 12-month follow-up assessments), before and after adding prerequisite orientation sessions to the trial protocol. A participant was considered retained if body weight was obtained at the primary outcome assessment.

Attendance at intervention sessions is presented by trial condition, before and after adding prerequisite orientations to the trial protocol. Three metrics are presented: mean percentage of sessions attended, percentage of participants who attended 0 sessions, and percentage of participants who attended ≥80% sessions. For SHINE, attendance was defined as participating in an in-person session. For Get Social, attendance was defined as exposure to an intervention module; that is, either attending an in-person, group-based session (Traditional) or by posting, replying, or liking a tweet during the same time period (Get Social). For GLOW, attendance was defined as participating in an individual intervention session (most conducted by telephone). As the GLOW intervention was compared with a usual care control group, there are no attendance data for the control condition.

Statistical Analysis

The impact of adding prerequisite orientations to trial protocols on retention and attendance was tested with multivariate models using estimation via generalized estimating equations. Regression models accounted for within–intervention group correlations where applicable (SHINE and Get Social) and within-person correlation among repeated measurements for valid estimation of treatment effects and associated SEs. Analyses examined if changes in retention and attendance before and after adding prerequisite orientations varied by trial condition using an interaction term (Model 1). If the interaction term was not statistically significant, a second model (Model 2) was conducted with the main effects.

For retention, the percentage of participants with available weight at primary outcome assessments was modeled with log-binomial regression. Given that approximately 10% of participants in Get Social provided a self-reported weight and were considered retained in the original trial, a sensitivity analysis was conducted for retention models that included and excluded participants with self-reported weights. There was no difference in outcomes between models (data not shown) and thus the model with all obtained weight data is presented here.

For attendance, mean attendance (continuous) was modeled with linear mixed effects, and the percentage of participants who attended 0 sessions and who attended ≥80% of sessions were modeled with log-binomial regression. To avoid estimation issues with zero-counts, simple Laplace smoothing was used by adding a pseudocount for models of attendance at 0 sessions.²⁰

Two sets of exploratory analyses were conducted for each of the 3 trials. First, to examine whether adding prerequisite orientations affected the proportion of eligible participants randomized, chi-square tests were used.

Second, to examine whether adding prerequisite orientations affected 5 key demographic characteristics of randomized participants, t-tests were used for the continuous variable (age), and chi-square tests for categorical variables (gender, education [4-year degree and above versus others], and race/ethnicity [non-Hispanic White versus others]). A priori criteria were set to evaluate if a difference for each demographic characteristic was substantively large enough to raise concerns: >5-year difference in mean age and >10% difference in percentage non-White, percentage who completed a 4-year degree, percentage female (where applicable), and percentage with BMI >30 kg/m² (where applicable; pre-pregnancy weight for GLOW). To adjust for multiple comparisons across the 5 demographic characteristics, a Bonferroni correction of p<0.05/5 = 0.01 was used.

All models assumed p<0.05 for statistical significance unless otherwise noted. Models were run in SAS, version 9.4 or and Stata, version 16.

RESULTS

To orient readers to the trial results (Table 3), unadjusted estimates of retention and attendance before and after adding prerequisite orientations and by trial condition are in Columns 2–8 and adjusted estimates of retention and attendance from the regression models examining the main effects of adding prerequisite orientations and by trial condition (i.e., Model 2) are in Columns 9–11. The impact of adding orientations did not differ by trial condition (Model 1 interaction term, all p> 0.05), except for Get Social attendance at 0 sessions; thus, for parsimony, Model 1 results are available upon request.

Table 3.

Trial Retention and Intervention Session Attendance Before and After Adding Prerequisite Orientations Sessions to Screening Protocols

Variables	Unadjusted estimates							Estimates, regression model 2
	Beforea (no orientation sessions)			Afterb (orientation sessions)			Differencec (after-before)	Beforea (no orientation sessions)	Afterb (orientation sessions)	Differencec (after-before)
	Unadjusted estimate	Unadjusted estimate	Unadjusted estimate (95% CI)	Unadjusted estimate	Unadjusted estimate	Unadjusted estimate (95% CI)	Unadjusted difference (95% CI)	Adjusted estimate (95% CI)	Adjusted estimate (95% CI)	Adjusted difference (95% CI)
	Trial condition		All befored	Trial condition		All aftere	All total			All total
	Control	Intervention		Control	Intervention
Retention
SHINE	Active control	Mindfulness	−	Active control	Mindfulness	−	−	−	−	−
Sample size, n	41	44	85	53	56	109	194	85	109	194
% weighed at 6 months	63.4%	77.3%	−	86.8%	89.3%	−	−	−	−	−
% weighed at 18 months	58.5%	75.0%	−	81.1%	85.7%	−	−	−	−	−
% weighed, collapsed	−	−	76.5% (67.5, 85.5)	−	−	88.1% (82.0, 94.2)	11.6% (0.7, 22.5)	68.8%f (59.0, 78.6)	86.1%g (79.6, 92.6)	17.3%h (5.5, 29.1)
Get Social	Traditional	Get social	−	Traditional	Get social	−	−	−	−	−
Sample size, n	36	39	75	71	71	142	217	75	142	217
% weight obtained at 6 months	80.6%	82.1%	−	94.4%	91.6%	−	−	−	−	−
% weight obtained at 12 months	77.8%	89.7%	−	94.4%	95.8%	−	−	−	−	−
% weight obtained, collapsed	−	−	82.7% (74.1, 91.3)	−	−	94.0% (90.1, 97.9)	11.3% (1.9, 20.7)	82.7%f (74.1, 91.3)	94.1%g (90.3, 97.9)	11.4%h (2.0, 20.8)
GLOW	Usual care	Getting in balance	−	Usual care	Getting in balance	−	−	−	−	−
Sample size, n	79	81	160	115	114	229	389	160	229	389
% assessed at 32 weeks’ gestation	94.9%	87.7%	−	96.5%	93.9%	−	−	−	−	−
% assessed at 6 months postpartum	96.2%	86.4%	−	96.5%	94.7%	−	−	−	−	−
% assessed at 12 months postpartum	97.5%	85.2%	−	94.8%	94.7%	−	−	−	−	−
% assessed, collapsed	−	−	94.4% (89.6, 97.4)	−	−	97.4% (94.4, 99.0)	3.0% (−1.0, 8.1)	91.9%f (87.1, 95.5)	95.6%g (92.9, 98.3)	3.7%h (−1.3, 8.7)
Attendance
SHINE	Active control	Mindfulness	−	Active control	Mindfulness	−	−	−	−	−
Sample size, n	41	44	85	53	56	109	194	85	109	194
Total number of sessions	17	17	−	17	17	−	−	−	−	−
Mean % of sessions attended	75.3%	71.5%	73.4% (68.2, 78.6)	77.9%	77.2%	77.5% (73.0, 82.1)	4.2% (−2.6, 11.0)	73.4%i (68.3, 78.5)	77.6%j (73.0, 82.1)	4.2%k (−2.7, 11.0)
% attended 0% of sessions	0.0%	0.0%	0.0% (0.0, 0.2)	1.9%	0.0%	0.9% (0.0, 2.7)	0.9% (−0.9, 2.7)	N/A	N/A	N/A
% attended >80% of sessions	58.5%	52.2%	55.3% (44.7, 65.9)	66.0%	60.7%	63.3% (54.3, 72.4)	8.0% (−5.9, 21.9)	55.4%f (44.8, 66.0)	63.4%g (54.4, 72.4)	8.0%h (−5.9, 21.9)
Get Social	Traditional	Get social	−	Traditional	Get social	−	−	−	−	−
Sample size, n	36	39	75	71	71	142	217	75	142	217
Total number of sessions	22	22	−	22	22	−	−	−	−	−
Mean % of sessions attended	38.9%	69.0%	54.5% (46.0, 63.1)	56.7%	69.8%	63.2% (57.9, 68.5)	8.7% (−0.8, 18.2)	54.2%i (46.8, 61.6)	63.2%j (57.8, 68.6)	9.1%k (−0.1, 18.2)
% attended 0% of sessions	33.3%	0.0%	16.0% (7.5, 24.5)	7.0%	2.8%	4.9% (1.3, 8.5)	−11.1% (−20.3, −1.9)	14.6%f,l (7.1, 22.1)	4.5%g,l (0.6, 8.4)	−10.1%h,l (−18.6, −1.6)
% attended >80% of sessions	13.9%	48.7%	32.0% (21.2, 42.8)	26.8%	52.1%	39.4% (31.3, 47.6)	7.4% (−6.0, 20.9)	29.8%f (20.1, 39.5)	38.4%g (30.4, 46.4)	8.6%h (−4.0, 21.2)
GLOW	Usual care	Getting in balance	−	Usual care	Getting in balance	−	−	−	−	−
Sample size, n	N/A	81	−	N/A	114	−	81	114	195	195
Total number of sessions	−	13	−	−	13	−	−	−	−	−
Mean % of sessions attended	−	82.2%	−	−	91.0%	−	8.8% (−0.03, 17.6)	82.2%i (76.0, 88.5)	91.0%j (85.7, 96.3)	8.8%k (0.6, 17.0)
% attended	−	N/A	-	-	N/A	N/A	N/A	N/A	N/A	N/A
0% of sessions
% attended >80% of sessions	−	79.0%	−	−	88.6%	−	9.6% (−0.9, 21.0)	79.0%f (70.1, 87.9)	88.6%g (82.8, 94.4)	9.6%h (−1.0, 20.2)

Notes: Boldface indicates statistical significance (p<0.05).

^aParticipants enrolled before prerequisite orientations were added to trial.

^bParticipants enrolled after prerequisite orientations were added to trial.

^cDifference between before and after prerequisite orientations were added to trial, collapsed across trial conditions and assessment visits.

^dUnadjusted estimate for all participants enrolled before prerequisite orientations were added to trial, collapsed across trial conditions and assessment visits.

^eUnadjusted estimate for all participants enrolled after prerequisite orientations were added to trial, collapsed across trial condition and assessment visit.

^fAdjusted estimated proportion for all participants enrolled before prerequisite orientations were added to trial (multiplied by 100%) from logistic regression Model 2.

^gAdjusted estimated proportion for all participants enrolled after prerequisite orientations were added to trial (multiplied by 100%) from logistic regression Model 2.

^hDifference between adjusted estimated proportions before and after prerequisite orientations were added to trial, collapsed across trial condition and assessment visit.

ⁱAdjusted estimated proportion for all participants enrolled before prerequisite orientations were added to trial, from linear regression Model 2

^jAdjusted estimated proportion for all participants enrolled after prerequisite orientations were added to trial, from linear regression Model 2.

^kDifference between adjusted estimates before and after prerequisite orientations were added to trial, collapsed across trial condition and assessment visit, from linear regression Model 2.

^lAnalyses comparing participants who attended 0% of sessions used simple Laplace smoothing (see Methods for details).

SHINE, Supporting Health by Integrating Nutrition and Exercise; GLOW, Gestational Weight Gain and Optimal Wellness.

After adding prerequisite orientations, all 3 trials attained higher participant engagement for trial retention or intervention attendance.

Trial retention at follow-up assessments was higher after adding prerequisite orientations in 2 trials (Table 3, adjusted estimates of differences, highlighted far right column): 17.3% (95% CI=5.5, 29.1) in SHINE and 11.4% (95% CI=2.0, 20.8) in Get Social.

Mean percentage attendance at intervention sessions was higher after adding prerequisite orientations in 1 trial (Table 3, adjusted estimates of differences, highlighted far right column): 8.8% (95% CI=0.6, 17.0) in GLOW.

In 1 trial, fewer participants attended 0 intervention sessions after adding prerequisite orientations: −10.1% (95% CI= −18.6, −1.6) in Get Social; SHINE and GLOW models did not converge owing to the low frequency of attending 0 sessions. In Get Social, there was also a greater decrease in attendance at 0 sessions in the Traditional (control) group than in the Get Social (intervention) group after adding prerequisite orientations (Model 1). Across the 3 trials, there was no change in attendance at ≥80% of intervention sessions after adding prerequisite orientations.

In all trials, after adding prerequisite orientations, there were no changes in the proportion of eligible participants randomized and no differences in demographic characteristics (Table 4). All 95% CIs included 0, even without applying the a priori multiple testing correction. Descriptively, across demographic characteristics, observed differences were inconsistent in direction across trials, and were small, not clinically relevant, and smaller than the a priori difference criteria.

Table 4.

Proportion of Participants Randomized and Demographic Characteristics Before and After Adding Orientations Sessions to Screening Protocols

Demographic characteristics	SHINE (N=194)		Get Social (N=217)				GLOW (N=389)
	Beforea (no orientation sessions)	Afterb (orientation sessions)		Beforea (no orientation sessions)	Afterb (orientation sessions)		Beforea (no orientation sessions)	Afterb (orientation sessions)
	Mean (SD) or N (%)	Mean (SD) or N (%)	Difference (95% CI)	Mean (SD) or N (%)	Mean (SD) or N (%)	Difference (95% CI)	Mean (SD) or N (%)	Mean (SD) or N (%)	Difference (95% CI)
Proportion eligible participants randomized, N randomized/N eligible (%)	85/93 (91.4%)	109/125 (87.2%)	−4.2% (−12.4, 4.0)	75/91 (82.4%)	142/194 (73.2%)	−9.2% (−19.0, 0.7)	160/199 (80.4%)	229/271 (84.5%)	4.1% (−2.8, 11.4)
Sample size	85	109		75	142		160	229
Age, years	45.1 (12.9)	48.5 (12.4)	3.4 (−0.2, 7.0)	44.6 (11.4)	45.5 (11.0)	1.0 (−2.2, 4.1)	32.8 (4.4)	32.3 (4.1)	−0.5 (−0.4, 1.3)
Race/ethnicityc			−1.3% (−15.2, 12.6)			4.2% (−0.4, 8.8)			−2.9% (−12.4, 6.6)
White	51 (60.0%)	64 (58.7%)		65 (86.7%)	129 (90.8%)		55 (34.4%)	72 (31.4%)
Non-White	34 (40.0%)	45 (41.3%)		10 (13.3%)	13 (9.2%)		105 (65.6%)	157 (68.6%)
Black/African American	11 (12.9%)	15 (13.8%)		1 (1.3%)	2 (1.4%)		14 (8.8%)	18 (7.8%)
Asian	10 (11.8%)	10 (9.2%)		1 (1.3%)	0 (0.0%)		30 (18.8%)	49 (21.4%)
Hispanic	9 (10.6%)	15 (13.8%)		5 (6.7%)	7 (4.9%)		32 (20.0%)	46 (20.1%)
>1 race/ethnicity; other race	4 (4.7%)	5 (4.6%)		6 (8.0%)	5 (3.5%)		29 (18.1%)	44 (19.2%)
4-year college degree & above	55 (65.5%)	70 (64.2%)	−1.3% (−14.8, 12.3)	49 (65.3%)	83 (58.5%)	−6.8% (−20.3, 6.7)	115 (71.9%)	167 (72.9%)	1.0% (−8.0, 10.1)
Female	72 (84.7%)	83 (76.2%)	−8.5% (−19.6, 2.5)	63 (84.0%)	113 (79.6%)	−4.4% (−15.0, 6.2)	160 (100%)	229 (100%)	N/Ad
BMI ≥30 kg/m2	85 (100%)	109 (100%)	N/Ae	68 (90.7%)	122 (85.9%)	−4.8 (−13.5, 3.9)	59 (36.9%)	80 (34.9%)	−1.9% (−11.6, 7.8)

Notes: Boldface indicates statistical significance (p<0.05).

^aParticipants enrolled before prerequisite orientations were added to trial.

^bParticipants enrolled after prerequisite orientations were added to trial.

^cDifference and CIs for change in % White.

^dGLOW enrolled a sample of 100% pregnant women.

^eSHINE enrolled a sample of 100% participants with BMI >30 kg/m².

SHINE, Supporting Health by Integrating Nutrition and Exercise; GLOW, Gestational Weight Gain and Optimal Wellness.

DISCUSSION

Across 3 trials with early signs of suboptimal and/or differential participant engagement, there was higher trial retention and intervention attendance after adding prerequisite orientation sessions. Each trial creatively and effectively adapted the MMI approach, such as the original in-person prerequisite orientations¹ to their specific needs. For instance, Get Social delivered orientations via interactive webinars and facilitated the pro/con discussions via webchat,¹⁷ whereas GLOW delivered orientations via telephone conference calls. Importantly, across the 3 trials, there were no changes in the proportion of eligible participants randomized or substantive differences in demographic characteristics after adding prerequisite orientations.

The current analyses are consistent with a recent pragmatic intervention trial for chronic pain patients that implemented the MMI approach of prerequisite orientations after early suboptimal patient engagement.¹³ Trial retention and attendance were higher for chronic pain patients after adding orientations, despite pain-related challenges for patients leaving their homes for in-person orientations and classes. Although substantially fewer eligible patients enrolled in the pragmatic trial, perhaps owing to adding a prerequisite orientation requirement to the enrollment process, researchers in the pragmatic trial offered that a parallel change in the health system’s policy for access to pain medications, external to the pragmatic trial, could have substantially lowered enrollment.¹³

Details about the 4 core constructs, key strategies, originally implemented procedures, and subsequent adaptations of the MMI approach are in Table 1. A critical consideration for investigators is to determine which comparison to present when discussing pros/cons. This discussion is likely to be most effective if potential participants are explicitly confronted with the most difficult comparison they will encounter during enrollment, such as whether to participate in a trial with considerable participant burden due to complex trial assessments or relative attractiveness between trial conditions. For instance, in Get Social, the key comparison for young adults to consider was between a Twitter-based program or a standard, in-person, group-based intervention.¹⁷ Given comparisons are made explicit, investigators need to thoughtfully select a control condition as well as transparently provide the scientific rationale for such conditions during orientations.²¹

When adapting and implementing prerequisite orientations for future trials, 3 sets of factors should be considered: logistical (research design and delivery, trial setting, and geographic region), participant (demographics or other participant characteristics likely to impact trial enrollment such as health status), and administrative (staff resources). However, across the trials here, upfront investment of staff resources was related to higher participant engagement. Given orientations can conveniently leverage innovative formats such as webinars or conference calls, orientations may efficiently reduce overall staff time and be cost effective.

The full impact of the MMI approach remains to be experimentally tested. Three potential avenues exist. One avenue could experimentally test specific MMI strategies rather than the entire approach. ²²,²³ For example, in online experiments informed by MMI, individuals who read an easy-to-understand, visually powerful, 1-page infographic letter illustrating the detrimental impact of dropouts on trial conclusions had substantially greater research literacy and participant trust in the research team relative to individuals reading a control letter.²² A second avenue could leverage optimization trial designs such as the Multiphase Optimization Strategy (MOST) framework²⁴ to identify which core MMI constructs and key strategies—or which combinations of these best enhance participant engagement. A third avenue could embed experiments testing the efficacy of MMI core constructs and key strategies²⁵,²⁶ within large randomized parent trials, similar to embedded recruitment experiments.²⁷ Embedded experiments could assess potential mediators such as participant trust in the research team as mentioned above or other posited measures in the broader retention literature, such as whether individuals feel respected and sufficiently informed about the trial procedures.²⁸

Limitations

Several study limitations exist. One was the reliance on a convenience sample of 3 trials that explicitly chose to add prerequisite orientations to their protocols in response to early suboptimal participant engagement. Though limiting in scope, this sample accessed granular trajectories of trial data (i.e., retention and adherence by cohort before and after implementing orientations), strengthening the analytic design. As more trials make data publicly available, additional trajectory analyses, including specific adjustments to improve these outcomes, will be possible.

A second limitation was the relatively small cell sizes for comparisons by trial condition and by prerequisite orientations, the latter in part because the 3 trials indeed moved quickly to respond to early suboptimal engagement. Yet, descriptively, all retention and attendance outcomes across all 3 trials were in the expected and higher direction, including modest-to-large, albeit non-significant, increases. Given the consistent pattern across outcomes, it is important to ensure not only uniform fidelity for trial interventions, but also fidelity for prerequisite orientations, and may be especially critical for smaller randomized trials.²⁹,³⁰

A third limitation was that the pre–post analytic design limited a causal determination. Indeed, implementing the prerequisite orientations in response to early signs of suboptimal engagement could have been accompanied by co-occurring adjustments to trial procedures. Yet, these adjustments may not be cofounding effects. Rather, the MMI approach provides a structure for open communication and actively listening of participants, thus providing investigators the opportunity to make proactive and responsive adjustments to ensure participant barriers are addressed quickly (e.g., schedule changes or parking stipends).

A fourth limitation is that the MMI approach could conceivably limit sample generalizability for trials by screening out less committed potential participants prior to randomization.³¹ Yet, there were no consistent or clinically relevant differences in the proportion of eligible participants enrolled or in demographic characteristics after adding orientations. Future research can examine additional baseline characteristics moderating the impact of the MMI approach, including lack of motivation to change the target behaviors or enrollment commitment. Alternatively, the MMI approach could intriguingly be applied to enhance trial generalizability by deeply engaging diverse and traditionally underserved populations, and proactively supporting participant buy-in to the scientific rationale of the trial with a transparent and comprehensive view of the commitment involved in trial participation well before enrolling.

Current study strengths included trials across 3 different intervention delivery channels (in person, social media, telephone) and 3 innovative adaptations to orientation format (in person, webinar/webchat, conference calls). Additionally, participant samples across trials were diverse in age and race/ethnicity.

CONCLUSIONS

The MMI approach, which integrates prerequisite orientation sessions, shows promise for increasing the rigor of trials by improving retention at follow-up assessments and intervention attendance. The MMI approach is readily adaptable to innovative in-person, webinar/webchat, and conference call formats. Future experimental research, embedded within ongoing parent trials, can strengthen the evidence base by examining MMI effects on retention, attendance, and generalizability of trial samples, and participant trust in the research enterprise.