Evaluation of Intervention Components to Maximize Outcomes of Behavioral Obesity Treatment Delivered Online: A Factorial Experiment Following the Multiphase Optimization Strategy Framework

Abstract

Background

Behavioral lifestyle intervention (BLI) is recommended as a first-line treatment for obesity. While BLI has been adapted for online delivery to improve potential for dissemination while reducing costs and barriers to access, weight losses are typically inferior to gold standard treatment delivered in-person. It is therefore important to refine and optimize online BLI in order to improve the proportion of individuals who achieve a minimum clinically significant weight loss and mean weight loss.

Study Design

Five experimental intervention components will be tested as adjuncts to an established 12-month online BLI: virtual reality for BLI skills training, interactive video feedback, tailored intervention to promote physical activity, skills for dysregulated eating, and social support combined with friendly competition. Following the Multiphase Optimization Strategy (MOST) framework, the components will first be refined and finalized during Preparation Phase pilot testing and then evaluated in a factorial experiment with 384 adults with overweight or obesity. A priori optimization criteria that balance efficacy and efficiency will be used to create a finalized treatment package that produces the best weight loss outcomes with the fewest intervention components. Mediation analysis will be conducted to test hypothesized mechanisms of action and a moderator analysis will be conducted to understand for whom and under what circumstances the interventions are effective.

Conclusion

This study will provide important information about intervention strategies that are useful for improving outcomes of online BLI. The finalized treatment package will be suitable for testing in a future randomized trial in the MOST Evaluation Phase.

1. Introduction

United States national guidelines recommend behavioral lifestyle intervention (BLI) as the first-line treatment for obesity given that BLI routinely produces clinically significant weight losses and corresponding improvements in comorbidities including cardiovascular disease, respiratory problems, and cancer.^1,2 BLI typically achieves these outcomes by fostering improvements in diet and physical activity patterns via behavioral strategies such as goal-setting, self-monitoring, self-reinforcement, stimulus control, and problem-solving.³ Despite repeatedly demonstrating efficacy in clinical trials, BLI has not been disseminated widely due to the high cost of treatment, which usually involves group and/or invidual treatment sessions led by specially-trained interventionists, over many weeks or months.³

Automated online programs have clear potential for improving the scalability, reach, and cost of BLI.⁴ Empirically validated online BLI typically involves some combination of online skills training, self-monitoring, feedback, and/or other electronic tools and resources to facilitate behavior change.³ While online BLIs routinely produce superior outcomes compared to control conditions in randomized controlled trials (RCTs), the weight losses often appear suboptimal.⁵ Whereas BLI delivered in-person produces mean weight losses of 7-10% of initial body weight, 3-5% is more common in online BLI.^3,5 Similarly, approximately 62-68% of participants may achieve a minimum clinically significant weight loss of ≥3-5% in BLI delivered in-person,^6,7 whereas 20-50% is more common in online BLI.^8–10 Comparable outcomes have been achieved via an online BLI developed by our research team. The 12-week Rx Weight Loss (RxWL) program, which was initially designed for provision to patients with overweight/obesity by physicians in the primary care setting,¹¹ consistently produces mean weight losses of approximately 5% of initial body weight across various contexts including community health campaigns¹² and worksite wellness programs.¹³ About half of participants achieve a minimum clinically significant weight loss of 5% of initial body weight. ^11–13

To maximize the public health impact of online BLI, research must be conducted to improve treatment outcomes, particularly the proportion of patients who achieve a minimum clinically significant weight loss and mean weight loss. This paper describes the rationale and design of a study to optimize RxWL and its outcomes. The research is informed by the Multiphase Optimization Strategy (MOST),^14,15 which involves systematic methods for efficiently optimizing treatment outcomes by determining which intervention components should be included in a comprehensive treatment package based on prespecified criteria. Within this framework, five experimental intervention components will be tested as potential additions to the preexisting RxWL loss program. These intervention components target known limitations of online BLI (e.g., suboptimal engagement and adherence, lack of experiential learning opportunities) by using behavioral theory to: (a) provide experiential training in behavioral skills via an online virtual reality (VR) program, (b) improve diet via video feedback, (c) promote physical activity via a tailored plan with home-based exercises, (d) address dysregulated eating behavior via training in Acceptance and Commitment Therapy (ACT) strategies, (e) provide social support and opportunities for friendly competition via an online community. The result of this study will be an optimized RxWL program that can be tested against the preexisting program in a future RCT.

2. Methods

2.1. Study aims and design

This study, sponsored by the National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK), aims to optimize online BLI by testing whether any of five theory-driven experimental intervention components increase the proportion of individuals achieving a minimum clinically significant weight loss of ≥ 5% of initial body weight and/or mean weight loss after 12 months of treatment. The rationale and design of the study methods are informed by the 3-Phase MOST framework.^14,15 Consistent with the MOST Preparation Phase, the experimental intervention components will first be tested with a small number of participants to identify any problems related to implementation. They will then be refined to maximize acceptability and feasibility prior to large-scale testing. Per the MOST Optimization Phase, a factorial experiment will then be conducted to determine which, if any, of the experimental intervention components contributes to improved treatment outcomes.¹⁶ Adults with a body mass index (BMI) in the overweight or moderately obese range (25-50 kg/m²) will receive the preexisting 12-month RxWL and a random combination of 0-5 of the experimental intervention components. A priori optimization criteria will be used to evaluate which, if any, of the experimental intervention components should be included in an optimized RxWL treatment package, which can then be tested in a future randomized controlled trial per the MOST Evaluation Phase. Mediation analysis will also be conducted to determine whether the experimental intervention components exerted their intended effects on their putative behavioral and psychosocial mechanisms, and whether those effects were associated with improved weight loss outcomes. Moderators analysis will explore for whom and under what circumstances the experimental intervention components might have been more or less effective. Assessments of weight and hypothesized mediators will occur at baseline and 3, 6, and 12 months after randomization. The study aims are:

2.1.1. Aim 1:

Refine and finalize the experimental intervention components with 3-month Preparation Phase testing.

2.1.2. Aim 2:

Optimize RxWL via a factorial experiment to determine which combination of intervention components maximizes the proportion of participants achieving a ≥5% weight loss and mean weight loss at 12-months.

2.1.3. Aim 3:

Conduct mediation analysis using proximal outcomes to test hypothesized mechanisms of action and moderator analysis to understand for whom and under what circumstances the interventions are effective.

2.2. The preexisting RxWL intervention.

RxWL is an automated online BLI that was initially developed to support weight loss in adult primary care patients with overweight/obesity.¹¹ First developed as a 3-month program, it demonstrated effectiveness in worksites¹³ and a community wellness campaign,¹² in addition to primary care. RxWL was recently enhanced to include a total of 12 months of intervention content as part of a pragmatic trial in a large primary care practice network.¹⁷

All participants begin the program with an automated online “kickoff” session that explains BLI basics including goal-setting and self-monitoring. Participants are encouraged to lose 0.5 kg to 1 kg per week, by limiting energy intake to 1200 kcal to 1500 kcal per week depending on starting body weight, to achieve a total weight loss of ≥ 10% of starting body weight. Other goals include consumption of a low-fat or Mediterranean diet (decided by the participant) and regular moderate-to-vigorous physical activity (MVPA) with weekly MVPA goals that gradually increase to 150 minutes per week by 3-months and 250 minutes by 12 months.

After the kickoff session, participants are provided with weekly interactive multimedia lessons on key BLI topics for 3 months, followed by 9 months of monthly lessons that are designed to support further weight loss and weight loss maintenance. Lessons highlight strategies from the Diabetes Prevention Program and Look AHEAD trial BLIs, with topics that include consuming a healthy diet, meal planning, building an exercise habit, restaurant eating, social support, and weight regain prevention, among others. The lessons involve a combination of animated graphical content with voiceover and live action video. The lessons are self-paced and allow participants to explore additional information on key topics of interest (e.g., additional examples of how to calculate the calorie content of various prepackaged foods in a lesson explaining label reading and portion size). Navigation through each lesson is self-paced and they are designed to require about 15 minutes to complete.

In addition to the lessons, participants are asked to self-monitor their total energy intake (kcals), daily MVPA minutes, and body weight for the first 3 months of the program and to enter these data on the RxWL platform at least weekly. They are asked to weigh immediately upon waking in the morning, in light or no clothing, with the scale placed on a hard surface. Participants are given instructions and resources for using paper diaries and online/mobile tracking apps for this purpose. Flexibility in the method of self-monitoring improves the potential for reach and dissemination of the program given that that some individuals may not own a smartphone for use with tracking apps and/or may have limited confidence in using tracking apps, a more complex technology. Participants are not encouraged to use one method or app over the others, but if the FitBit™ app is used for self-monitoring, the data can be shared automatically with RxWL via an application programming interface (API). During the final 9 months of the program, participants are asked to self-monitor for one week per month, coinciding with the provision of the monthly lesson. At the end of each week of self-monitoring, a feedback message is algorithmically generated based on participants’ self-reported weight, daily reported energy intake (kcals), and daily reported exercise minutes. The content of the feedback changes weekly and provides encouragement and constructive feedback to support adherence to BLI strategies taught in the lessons. The RxWL platform also contains supplementary printable guides on topics such as eating healthfully on a budget, exercising safely, meal planning, and problem-solving. Finally, reminders to engage with the program are sent weekly to participants who have not logged their self-monitoring data.

2.3. Rationale and description of the experimental intervention components.

The MOST framework helps to guide identification of intervention components to be evaluated in treatment optimization research, in part via principles based in engineering. In particular, MOST emphasizes testing intervention components that are expected to have a positive effect on a distal health outcome (in this case, body weight) via proximal outcomes (i.e., mechanisms/mediators) that are theoretically driven, varied, and assessed as part of the research.¹⁵ This approach helps to determine not only whether intervention components are effective, but also why (i.e., because they did or did not have the expected effect on the proximal outcome and/or because the proximal outcome did or did not affect the distal outcome). MOST also emphasizes the resource management principle, which involves making best use of the available resources (e.g., time, money, participant and researcher effort) to most quickly and effectively advance science and clinical care.¹⁵ It follows from this principle that the intervention components tested in optimization research should have strong preliminary evidence and/or rationale for improving treatment outcomes. Finally, the continuous optimization principle emphasizes the importance of continuous, iterative, efforts to improve treatment outcomes, and building on an established base of efficacy by testing and integrating novel intervention strategies.¹⁵

Four key criteria, informed by principles of the MOST framework described above, drove selection of the intervention components to be tested in this study. Criterion #1: Intervention components must address known barriers to weight loss via training in skills with a strong theoretical foundation that are proven for enhancing weight loss. Criterion #2: Intervention components must have demonstrated feasibility and acceptability (primarily in BLI delivered in-person). Criterion #3: Intervention components must be innovative, both in the sense that they have not already been tested extensively in online BLI, and the method of implementing them is innovative. Specifically, we focused on methods involving experiential training (skills training and practice versus didactics). The way that each intervention component meets the 3 criteria is discussed in the description of each intervention component below. Criterion #4: Intervention components must involve minimal additional incremental cost per participant in order to maximize potential for scalability and dissemination. This resulted in a focus on automated online resources for skills training and support, as opposed to more costly physical resources such as meal replacements, human-delivered intervention, or activity tracking technology.

Figure 1 summarizes the purported mechanisms for the intervention components to be tested, including weight loss self-efficacy, implementation of dietary skills, accountability, social support, disinhibited eating, and performance of moderate-to-vigorous physical activity (MVPA). Evidence for each mechanism is presented in the description of each intervention component below. While each intervention could be expected to have an effect on any of the identified mechanisms, we focus on the pathways by which the theoretical foundations of each intervention lead us to expect the strongest effects. Figure 2 illustrates the schedule of key activities within the intervention components, which were timed to reduce overlap and thus limit burden for participants assigned to receive multipole components.

Figure 1.

The experimental intervention components to be tested in this trial and their purported behavioral and psychosocial mechanisms to be tested in mediators analysis.

MVPA = moderate to vigorous intensity physical activity

Figure 2.

Schedule of key intervention activities for the core program and experimental intervention components.

Note. Gray shading indicates that the component is active and available. A = Self-assessment, F = Feedback delivered, G = Goal setting with tailored plan, Gray shading indicates that the component is active and available. A = Self-assessment, F = Feedback delivered, G = Goal setting with tailored plan, L = Online lesson, M = Selection of lesson module and first lesson, S = Self-monitoring, VR = New virtual reality module becomes available.

2.3.1. Online VR intervention for training in basic behavioral weight loss skills.

Much of BLI’s effectiveness is attributed to training in basic skills such as self-monitoring, identification and consumption of low-calorie foods, regular performance of MVPA, stimulus control (changing internal and external cues of eating and physical activity behavior to promote healthy choices), problem-solving, and assertiveness (to resist peer pressure and build social support).³ Decades of behavioral theory (primarily Social Cognitive¹⁸ and Self-regulation¹⁹ theories) and empirical research provide evidence for the importance of these skills. When BLI fails to produce optimal outcomes, online or in-person, insufficient adherence to these skills is often identified as the barrier.^3,20 Given the limitations of didactics (i.e., online lessons) for skills training in online BLI, an online VR system for experiential skills training was developed in previous research.^21,22 Founded in Social Cognitive and Self-regulation theories,^18,19 the experiential training involves seeing virtual actors (i.e., live humans superimposed on a virtual background) correctly demonstrate skills use, practicing skills in virtual environments, and receiving praise from virtual actors for correct skills use. Experiential skills training is designed to enhance confidence for implementing basic weight loss skills in real-world settings (i.e., self-efficacy, the proximal outcome of this intervention).^23,24

Experience Success is a fully automated online based VR intervention for experiential training in basic behavioral weight loss skills. Skills training occurs in virtual environments where challenges typically experienced by individuals losing weight (e.g., selecting from a variety of foods ranging in calories and healthfulness, peer pressure to overeat and be inactive, environmental cues for overeating, common barriers to physical activity) are presented. Participants see VR actors implementing skills, practice skills themselves, and receive praise for successful use of skills. Participants end each session by making explicit plans for real-world skills use. The intervention consists of four modules corresponding to four virtual environments: home, the workplace, the gym, and a party (i.e., a social eating situation). Each module contains about two hours of content, but based on a participant’s choices, a single scene playthrough takes about 20 minutes. An example of the challenges faced by users in the home module involves how to prepare and pack a healthy low-calorie lunch before the bus to work arrives. The user is presented with a selection of ingredients, cooking methods, and packaged foods. They then experiment with different combinations that balance caloric content, dietary quality, and preparation time until they finalize a meal, which they then learn to prepare. Experience Success was recently found to enhance 12-week weight loss outcomes of the commercial online Weight Watchers program.²²

2.3.2. Tailored interactive video feedback targeting personalized training in dietary skills.

Self-regulation theory posits that positive behavior change is best accomplished via goal setting, self-monitoring of progress towards those goals, problem-solving, and reinforcement. Feedback is an essential component of this process.¹⁹ It facilitates problem-solving and reinforcement, and it also increases accountability (i.e., responsibility for one’s choices and their results). Feedback is therefore also considered an essential component of behavior change interventions.²⁵ While live human feedback produces the best outcomes,²⁶ it is costly and limits scalability. This component therefore replaces the existing text-based feedback in the core RxWL program, with the video feedback delivered at the same intervals as the text-based feedback but via a video-recorded human. Given that feedback is most effective when it is accurate, this component also adds an interactive component to help better identify and address behavioral challenges.

Like the existing text feedback that this component replaces, the video feedback is automated and uses the exact same algorithm to assemble an aggregate message of 3-4 pre-recorded video clips (approximately 30 seconds each) that varied each week. Video feedback messages are delivered on-camera by an obesity treatment expert for a more interactive experience and to enhance a sense of human connectedness. Feedback content is nearly identical to the text version but modified for a more conversational tone. At the end of the feedback message, participants are given the opportunity to “interact” by choosing from a predetermined set of situations and behavioral challenges to receive a further refined feedback video. For example, participants may select that they are traveling or have experienced sickness, injury, or a significant life stressor. The corresponding feedback teaches skills such as healthy restaurant eating and stress management. If a participant indicates that they are traveling, they might be encouraged to order smaller portions (e.g., appetizers instead of entrees) at restaurants and look for opportunities to increase physical activity to offset increased intake. Additional feedback given in response to selecting a behavioral challenge generally falls into one of the following categories: behavioral skills to support healthy eating (e.g., meal planning), selecting and preparing healthy foods (e.g., increasing fruits and vegetables, reducing sugary beverage intake), maximizing accuracy of self-monitoring (e.g., portion size estimation), and physical activity adherence (e.g., enhancing motivation, introducing variety). Training in dietary skills is particularly emphasized as it provides a method of improving two key elements that drive the feedback, namely dietary intake and weight change. The options for interactivity are titrated each week to reflect the progressive nature of weight management skills acquisition and complement skills taught in the RxWL video lessons (i.e., more basic skills are emphasized earlier in the program, and more advanced skills later in the program).

2.3.3. Tailored interactive intervention to promote structured physical activity.

BLI emphasizes habitual MVPA,³ as it directly contributes to an energy deficit and has additional indirect physiological and psychological benefits including improved appetite regulation,²⁷ weight regain prevention,²⁸ and improved cognitive function which may improve overall treatment adherence.²⁹ While BLIs (including the core RxWL program) encourage setting physical activity goals and self-monitoring progress,³ most participants struggle to adopt and maintain the 250 min/wk of structured MVPA targeted by RxWL. This component targets common barriers to physical activity (lack of instruction, low self-efficacy) by helping the participant to establish appropriate goals and a plan for physical activity, and by providing home-based exercise videos that are tailored to the participant’s plan.

This component consists of three parts, including evaluation of physical activity, a tailored physical activity plan, and instructional and educational videos for home-based physical activity. The core RxWL program focuses on eating and dietary behavior in the firsts two weeks and then targets physical activity behavior beginning in the third week, which involves gradually increasing PA to 150 minutes during the first 3 months, then increasing to 250 minutes by the end of the 12-month program. Core program content focuses on gradually increasing PA to meet overall goals, and involves general suggestions for safely building a PA habit that emphasizes brisk walking (e.g., planning ahead for PA, stretching before and after PA).

Consistent with the core RxWL program structure, in week 3 participants receiving the experimental PA intervention component complete an online evaluation of physical activity, with a focus on time spent in structured MVPA. These data inform a tailored physical activity plan that promotes adding 50min/wk (+10 min/day on 5d/wk) to participants’ current level of physical activity each month until participants achieve the national guideline of MVPA for weight loss (250 structured minutes weekly).³⁰ Participants select their preferred plan (10, 20, 30, 40, or 50 min/day on ≥5 days weekly) for the upcoming four weeks. If participants feel strongly that their suggested goal is not appropriate for them, they can select a different goal (e.g., to push themselves harder or select a lower activity level). The physical activity evaluation, generation of the tailored activity plan, and delivery of tailored video content reoccurs after two weeks, and monthly thereafter for the duration of their participation.

Participants are given long-form exercise videos to complete at home. To ensure that participants have appropriate support for each of the physical activity goals they could select during the evaluation and planning step, videos are available with durations of 10, 20, 30, 40, and 50 minutes. The complexity and intensity of the videos increases slightly with duration, but all videos are designed to produce MVPA. Videos primarily utilize body weight exercises targeting all muscle groups and increasing heart rate. Each exercise is presented with modifications for adaptation to the participant’s functional capacity, preexisting injuries, and movement preferences. All the exercises can be done within a 3’×2’ space. No equipment is needed for most exercises, but a mat and light hand weights are utilized in some adaptations. Participants are given guidance on safety, avoiding injury, and alternatives to hand weights that they may use if desired. Most of the exercises are conducted over a time interval (e.g., “Do this exercise for 45 seconds”) so that participants do not need to count repetitions and sets of weight-bearing exercises. Each video includes background music, narration, and instructors synchronously demonstrating the same exercise from different views and/or at different ability levels. When the PA goal is selected, all videos at that duration and lower are made available to allow flexibility in meeting the goal (e.g., at participant with a 40 min/day goal may choose to do the 20 minute video twice in a day if they find it difficult to set aside 40 uninterrupted minutes for physical activity).

Participants are provided with supplementary videos each month in addition to the long-form videos. The topics are primarily instructional (e.g., upper body exercises, lower body exercises, warming up and cooling down, core mat work) or educational (e.g., overcoming exercise barriers, converting intentions into actions). Participants setting a higher MVPA goal (based on recent MVPA engagement) receive additional videos to facilitate additional optional strength training and exercise of higher intensities. Participants are instructed to count sets and repetitions (or use time goals when appropriate) and determine the appropriate supplementary weight needed. They are also given injury prevention information, tips for proper form, and suggestions for modifying the 10-50 minute videos to increase intensity. These videos are not made available to participants selecting a lower minute goal in order to avoid overwhelming them and to focus them on building an initial physical activity habit. Participants may access the supplementary videos available to them as often as they like.

2.3.4. Tailored interactive skills training to address dysregulated eating.

Dysregulated eating, particularly overeating that occurs in response to negative cognitive and/or emotional cues, is common and a known barrier to weight loss and weight loss maintenance.^31,32 Although most BLIs (including the prior “core” RxWL program) promote “controlling” or “changing” negative thoughts and emotions (e.g., distraction, thought stopping),³ recent studies suggest that such control strategies may actually interfere with coping and lead to overeating.^33,34 Alternatively, cognitive-behavioral techniques such as Acceptance and Commitment Therapy (ACT) aim to promote behaviors that are consistent with personal values and life goals (e.g., health and weight loss) even when unwanted thoughts and feelings are present.³⁵ ACT has been shown to improve dysregulated eating behaviors and facilitate weight loss.³⁶

This component consists of interactive skills training videos (i.e., “modules”) that utilize ACT approaches to target the impact of dysregulated eating on weight loss. The ACT strategies are presented as compatible with the “control” and “change” strategies that are taught much more briefly in the core RxWL program; participants are encouraged to use ACT strategies when they find that the strategies taught in the core program are insufficient for addressing dysregulated eating. Starting in week 4, participants complete an evaluation to identify negative internal experiences (i.e., thoughts, feelings, and bodily sensations) that can trigger dysregulated eating by rating statements on a 5-point Likert scale (e.g., “When I’m upset, I use food to make myself feel better”, “I need to have fewer food cravings in order to succeed with my weight management”, “Excuses often get in the way of me meeting my health goals”). An algorithm then generates suggestions for modules to be completed based on participant responses and provides a brief summary of the content of each module. The participant selects a module, which consists of four consecutive weeks of 20-minute video lessons with assignments to practice skills in between sessions. There are four modules: 1) the “Cravings” module uses guided-meditation exercises to teach craving awareness and tolerance (i.e., the ability to non-judgmentally observe cravings without eating to make them go away); 2) the “Thoughts” module uses metaphors and experiential exercises to help participants recognize patterns of difficult thoughts and learn alternative strategies for coping with them (e.g., non-judgmental acceptance); 3) the “Acceptance” module uses metaphors and guided-mediation exercises to illustrate the consequences of trying to control emotional experiences (e.g., using eating to change negative emotions) and presents an alternative: being willing to feel undesirable emotions in the service of staying on track with one’s health goals; and 4) the “Motivation” module uses experiential exercises to clarify participant values and explicitly link these values to RxWL program goals (i.e., reducing calorie intake, improving diet quality), and then teaches strategies to strengthen commitment to one’s chosen values. The dysregulated eating evaluation and tailored suggestion process repeats monthly.

2.3.5. Online platform for social support and friendly competition.

Decades of research have highlighted the potential of social support^37–40 and friendly competition^41–44 to enhancing the outcomes of BLI and BLI delivered online. A study of RxWL in the context of a statewide physical activity and weight loss campaign showed that connecting with peers who share a weight loss goal improved weight loss outcomes.⁴⁵ Research suggests that social interactivity may be especially important for weight-loss maintenance in online BLI.⁴⁶ Furthermore, large numbers of individuals who lose weight via commercial online tools seek out opportunities to connect online with other individuals losing weight, particularly for the purposes of: (a) encouragement and motivation, (b) shared experience (i.e., sharing common goals, struggles, and experiences), (c) accountability, (d) friendly competition, (e) and recognition of success.⁴⁷

This component provides participants with the opportunity to post messages to each other in a shared public form. Conversation threads allow for in-depth conversation and discussion, while a Twitterlike feed allows participants to post short messages and questions. Participants may also create a profile, including a photo, their first name (or a preferred nickname to maintain anonymity), and a brief message about interests, preferred weight loss strategies, and challenges. Participants will be discouraged from discussing their experiences with the other experimental intervention components on the public form in order to limit risk of contamination between conditions. Communication in the online forum will be monitored and participants will be asked to remove any mention of other experimental intervention components, should they be posted.

Friendly competition, accountability, and recognition of success are facilitated by a leaderboard for weekly/total weight loss, weekly/total MVPA minutes, and longest streak of days for self-monitoring. The top 10 participants’ first name or nickname is displayed. Participants are also awarded badges that are displayed on their profile when they accomplish important milestones. Some can be reached quickly, such as losing the first 5 lbs., self-monitoring daily for a full week, and exercising for ≥50 mins within a single week. Conversely, some milestones require weeks or months to achieve, such as reaching a 5% and 10% weight loss, self-monitoring daily for a month, and exercising for a total of 5,000/10,000 minutes. When a new badge is earned, the participant receives an online notification.

2.4. The refinement phase pilot study.

While the experimental intervention components to be tested in this study are based on well-established behavioral theory and science and are expected to produce measurable change in their target constructs, they have never been combined and tested in a factorial experiment. Thus, the study includes an initial phase of pilot testing and refinement to address any problems with implementation of the experimental intervention components and study protocol prior to executing the larger trial described below. Thirty-two participants, representing one full block of the 32 patterns of assignment to the five intervention components to be used in the larger trial, will complete the initial 3-months of the full trial protocol described below. These participants will also complete questionnaires and a semi-structured interview aimed at identifying problems and potential solutions related to specific combinations of the experimental intervention components; all other pilot procedures are identical to the full trial. Any problems that are identified will be addressed prior to the larger trial.

2.5. The factorial experiment.

All participants in the proposed randomized experiment will receive access to 12 months of the RxWL intervention and in addition will be randomized to receive 0-5 of the experimental intervention components, using a full factorial design (Table 1).¹⁶ The majority of participants (63%) will receive two or three intervention components, 16% will receive one or four components, 3% will receive zero or five components. Importantly, each intervention component has been designed explicitly such that receiving any of the other intervention components is neither required nor precluded to fully experience the intervention component and obtain its full benefit. The full factorial design will allow tests of whether there are interactions between components in their effect on weight loss such that certain combinations produce an effect that is stronger or weaker than would be expected given the strength of the main effects.

Table 1.

Factorial Design with 2⁵ =32 Conditions for Testing Innovative Intervention Components Added to the Rx Weight Loss Intervention Delivered Online.

Experimental Condition	Intervention Component
	Virtual Reality for Skills Training	Tailored Interactive Video Feedback	Tailored Intervention for Structured Physical Activity	Skills Training for Dysregulated Eating	Platform for Social Support & Friendly Competition
1	No	No	No	No	No
2	No	No	No	No	Yes
3	No	No	No	Yes	No
4	No	No	No	Yes	Yes
5	No	No	Yes	No	No
6	No	No	Yes	No	Yes
7	No	No	Yes	Yes	No
8	No	No	Yes	Yes	Yes
9	No	Yes	No	No	No
10	No	Yes	No	No	Yes
11	No	Yes	No	Yes	No
12	No	Yes	No	Yes	Yes
13	No	Yes	Yes	No	No
14	No	Yes	Yes	No	Yes
15	No	Yes	Yes	Yes	No
16	No	Yes	Yes	Yes	Yes
17	Yes	No	No	No	No
18	Yes	No	No	No	Yes
19	Yes	No	No	Yes	No
20	Yes	No	No	Yes	Yes
21	Yes	No	Yes	No	No
22	Yes	No	Yes	No	Yes
23	Yes	No	Yes	Yes	No
24	Yes	No	Yes	Yes	Yes
25	Yes	Yes	No	No	No
26	Yes	Yes	No	No	Yes
27	Yes	Yes	No	Yes	No
28	Yes	Yes	No	Yes	Yes
29	Yes	Yes	Yes	No	No
30	Yes	Yes	Yes	No	Yes
31	Yes	Yes	Yes	Yes	No
32	Yes	Yes	Yes	Yes	Yes

2.5.1. Participant eligibility criteria.

Eligibility criteria are kept broad in order to select a sample representative of the target audience for dissemination of RxWL. Inclusion criteria include age 18 to 70 years old; BMI 25-50 kg/m²; access to the Internet; an ability to walk two city blocks without stopping; and English fluency and literacy at the 6^th grade level. Exclusion criteria include current participation in another weight loss program; current use of weight loss medication; weight loss of ≥ 5% of body weight in the previous 6 months; history of bariatric surgery; pregnancy within the previous 6 months; a plan to become pregnant within 12 months; report of a heart condition; chest pain during periods of activity or rest in the previous 6 months; loss of consciousness in the previous 6 months; report of a medical condition that could make participation in unsupervised physical activity unsafe; or report of a condition that would result in an inability to follow the study protocol including terminal illness, substance abuse, an eating disorder not including Binge Eating Disorder, and untreated major psychiatric illness.

2.5.2. Participant recruitment and enrollment.

Participants will be recruited via advertisements in local media (e.g. newspapers, radio); targeted online advertising (e.g., Facebook, Google AdWords); flyers and advertisements posted in waiting rooms and exam rooms in primary care offices used in previous studies of RxWL; informational materials made available as part of the health and wellness program for employees in the Lifespan Heath System and hospital network (an approach used in a previous RxWL trial); and direct mailings. A screener completed online or by phone will be used to determine initial eligibility. Individuals who appear eligible will be invited to attend an orientation session at the research center. The study will be described in detail and participants will have the opportunity to complete informed consent, begin the baseline assessment, and schedule their randomization visit. Participants are required to complete the baseline assessment, including wearing the physical activity monitor and completing dietary recalls (described below), to be eligible for randomization.

2.5.3. Randomization.

Prior to enrollment, a randomization schedule will be prepared based on 12 blocks of 32 patterns of assignment to each of the five intervention components. A sequentially-numbered sealed envelope containing the assignment will be opened by research staff at the time of randomization. The participant will receive brief written materials explaining their assignment and how to begin treatment via the online system. Further orientation to the treatment system is conducted online via interactive video lesson. Study staff will be available by phone to trouble-shoot technical difficulties, but no human-delivered weight loss counseling will be provided in-person or by phone. Participants will be enrolled until the total N=384 target enrollment has been satisfied.

2.6. Measures

Assessments will be conducted at baseline, 3, 6, and 12 months. The measures below will be administered by an assessor blinded to group assignment. Participants will receive $25 for completing the 3-month assessment and $50 for completing the 6 and 12-month assessments.

2.6.1. Primary Outcomes.

Weight will be measured to the nearest 0.1 kg using a digital scale; height will be measured to the nearest millimeter with a stadiometer using standard procedures. Measurements will be made in light indoor clothing without shoes. Percent weight loss will be calculated as: [(baseline weight – follow-up weight) ÷ baseline weight] × 100.

2.6.2. Mediators.

Figure 1 depicts the primary mechanistic targets of each intervention component, to be tested in mediation analysis. The mediators are described in detail below.

2.6.2.1. Weight loss self-efficacy.

Defined as confidence in implementing behavioral weight loss skills, it is targeted by the online VR intervention for training in basic behavioral weight loss skills and tailored interactive video feedback. It will be measured via the established Weight Efficacy Life-Style Questionnaire (WEL) that involves rating confidence in one’s ability to cope with 20 weight loss challenges on a scale of 0 (not confident) to 9 (very confident)⁴⁸ Reliability of the five subscales ranges from .70-.90 and scores improve with weight loss.

2.6.2.2. Dietary intake.

Dietary quality and dietary energy density will be measured at each assessment via three, nonconsecutive 24-hour diet recalls collected using the web-based Automated Self-Administered 24-hour recall system (ASA-24™) developed and validated by the National Cancer Institute.⁴⁹ The three recalls will be collected to reflect intake on two weekdays and one weekend day, which will provide a measure of usual intake. The ASA-24™ uses the USDA’s validated Multiple Pass Method to collect detailed information on the foods and beverages consumed at each eating occasion over the previous 24-hour period.⁵⁰ It provides comparable estimates to those obtained by interviewer-led recalls.^51,52 At each assessment, participants will attend a study visit at which they will complete the first 24-hour recall with the help of research staff. They will then be prompted by email to complete two subsequent recalls on random days (including one weekend day) within a 7- day period. Given the shortcomings of self-reported dietary intake in providing reliable estimates of total energy intake,^53,54 the objective of collecting these recalls is to assess adherence to dietary prescriptions set forth in the online interventions. Thus, the variables of interest include food group consumption (sugar-sweetened beverages, alcohol, energy-dense snack foods, and fast foods), overall diet quality measured by the Healthy Eating Index, 2015 (HEI-2015), and the energy density of the diet (total energy intake from food ÷ total weight of the food). The HEI-2015 total score provides an overall measure of diet quality which reflects conformance to the 2015 Dietary Guidelines for Americans (https://epi.grants.cancer.gov/hei/). The measure uses a density-approach and is comprised of 12 component scores which assess adherence to key dietary recommendations including adequacy of intake for total vegetables, greens and beans, total fruit, whole fruit, whole grains, dairy, total protein foods, seafood and plant proteins, fatty acid ratio and moderation of refined grains, sodium and percent energy from empty calories.⁵⁵

2.6.2.3. Accountability.

Defined as the implicit or explicit expectation that an individual may be called upon to justify his or her actions or inactions, accountability is targeted by tailored interactive video feedback and the platform for social support and friendly competition. It will be measured via the Supportive Accountability Questionnaire,⁵⁶ via which participants use a 7-point Likert scale to rate their perceptions that they are held accountable to somebody else for their adherence to the intervention. The 6-item measure has been used in previous research on online interventions for weight management, where it demonstrated good reliability (Cronbach alpha=.70).⁵⁷

2.6.2.4. Social support for diet and exercise behaviors.

Defined as positive comments, encouragement, participation, involvement and rewards from friends and family for diet and exercise behaviors, this construct will be measured by the Scales to Measure Social Support for Diet and Exercise Behaviors.⁵⁸ This questionnaire has been validated and used extensively. Over 43 items rated on 5-point scale indicating the frequency with which family and friends had engaged in supportive or critical behaviors described in the item during the previous 3 months (1, none to 5, very often), separate subscales measure friend and family support for eating and exercise, each demonstrating independence from the others, with good test-retest reliabilities (r=.55-.86) and internal consistencies (Cronbach alpha=.61-.91).

2.6.2.5. Internal disinhibition.

Defined as a tendency toward overeating in response to negative cognitive and/or emotional cues, it is targeted by skills training to address dysregulated eating and measured by the 16-item Internal Disinhibition subscale of the established and validated Eating Inventory, via which patients rate whether they have engaged in a variety of dysregulated eating behaivors.⁵⁹ Confirmatory factor analysis found a standardized item factor correlation of 0.77, Cronbach alpha of .77, and associations with weight change.^32,60

2.6.2.6. Structured physical activity.

Daily time spent in structured MVPA will be measured using the ActiGraph GT9X Link (AG), which includes a rigorously validated triaxial accelerometer and wear time sensor. The AG reliably measures PA in adults over seven days in free-living conditions.⁶¹ AG data will be processed using ActiLife software version 6.11.3. Raw data will be processed using the low-frequency extension filter and integrated into 60 sec epochs. Time spent in MVPA will be determined using published and validated hip-worn cut points.⁶² Participants will wear the AG on the hip during all waking hours for seven consecutive days at each assessment period. The display will show the time only. Participants must provide ≥5 d of data (including one weekend day), defined as ≥600 min during the hours of 7 am to 11 pm, to be included in analyses.⁶² Daily time spent in MVPA will be expressed as min/d and % of daily monitor wear time. Analysis will focus on total and bout-related MVPA (i.e. MVPA accumulated in bouts ≥ 10 min).

2.6.2.7. Engagement and adherence.

The online platform automatically logs all aspects of participant engagement (e.g., date/time of access, duration of access, use of specific tools/resources).

2.6.3. Moderators

2.6.3.1. Demographic characteristics.

Age, sex, race, ethnicity, level of educational attainment, household income, employment status, and marital status will be assessed via questionnaire at baseline.

2.6.3.2. Baseline values of the mediators.

Individuals with higher versus lower baseline scores on the mediators will be identified to determine whether degree of opportunity for improvement on intervention targets relates to treatment outcome.

2.6.4. Other Measures

2.6.4.1. Contamination among conditions.

In order to evaluate potential contamination among conditions, participants assigned to receive the online platform for social support and friendly competition will be asked if they discussed any of the other experimental intervention components with other participants via the online platform.

2.6.4.2. Unscheduled participant contact with research staff.

Participants may contact the study team outside of planned assessment visits to ask for technical support, weight loss counseling, or other help with other concerns. While the research team may provide brief technical support related to the intervention platform, they are forbidden from providing weight loss counseling or other forms of support. Thus, all interaction with research participants outside of scheduled assessment visits will be logged with the participant ID, date, reason for call, resolution (e.g., whether the technical issue was resolved), and duration of call. The study investigators will review these logs at regular intervals to ensure that staff are following rules for unscheduled participant contact.

2.7. Analytic plan, sample size, and power estimates

2.7.1. Analytic plan.

Statistical analysis will follow good practices for the evaluation of randomized controlled trials as embodied in the CONSORT statement.⁶³ Missing data will be imputed using a multiple imputation approach and outcome models averaged across imputations to adhere to the intent-to-treat principle. We will compare the sensitivity of the findings to alternative methods for handling missing data (see Missing Data Section below).

As a preliminary step, demographics, baseline weight and health history will be summarized across the aggregate sample and compared between cells (intervention components) using Analysis of Variance (ANOVA) for continuous variables, chi-squared analyses for categorical variables and non-parametric tests as appropriate. Variables will be considered potential confounders in the subsequent models if these variables are correlated with the outcome under consideration (at a modest p<.30 level). The distribution of each of study outcome will be assessed using both parametric and graphical methods and transformed as necessary (e.g. log transform towards normality). Potential distributions for the outcome variables include normal and binomial, and zero-inflated distributions.

Evaluation of the intervention components will involve generalized linear models that regress percent weight change from baseline (e.g., normal distribution & identity link function) and achievement of a ≥5% weight loss (binomial distribution & logit link function) on effect-coded indicators for each of the five intervention components and unique 2 through 5-way interactions of components. Effect coding (−1,1) is essential in analyzing factorial designs and results in regression weights that correspond to main effects and interactions. Effect-coded main effects and interactions are orthogonal to one another, yielding unbiased estimation of standard errors, and equal power for main effects and interactions for given effect sizes.¹⁶ Using this approach, individual cell sizes (for each combination of components) does not limit statistical power. Models will adjust for confounders identified during preliminary analyses, including demographics, health and weight history that are associated with weight loss outcomes.

We will use a multiple mediation approach, in which all potential mediators (e.g., weight loss self-efficacy, implementation of dietary skills, accountability, social support, internal disinhibition, physical activity) are tested simultaneously using a product of coefficients method with bootstrapped standard errors (5000 samples with replacement). We will estimate the path coefficients (a path: effects of the intervention component on changes in mediators over time and b path: effects of changes in the mediators on weight loss over time, controlling for baseline), as well as the indirect effect of treatment (ab path: effect of intervention components on weight loss through the mediators). Models will be run separately for each of the primary and secondary weight loss outcomes. Models will be specified using effect coding to allow for identifying mediators of each of the intervention components.

Lastly, we will explore additional potential moderators of the intervention component effect (e.g. sex; demographics including age, race, and ethnicity; baseline psychosocial factors) using a similar analytic approach as that described for the primary outcomes analysis. Specifically, models for each of the weight loss outcomes will additionally include main effects of the potential moderator, as well as all interactions between intervention components and the moderators. While the majority of participants (81%) will be randomized to 3 or fewer of the experimental intervention components, the number of intervention components assigned will also be tested as a potential moderator given that receiving a more complex intervention with a larger number of components could interfere with positive treatment outcomes. A variable will be considered a moderator if the component by moderator effect is significantly different than zero.

2.7.2. Missing data.

Analyses will be conducted on the intent-to-treat sample (everyone randomized will be included in the final analysis) under various assumptions about the missing data mechanism. Sensitivity to these assumptions will be tested. Specifically, we will gather follow-up information and reasons for dropout regardless of protocol completion and censor at the point of loss. We will compare the robustness of our findings using three statistical approaches for handling missing data. First, we will use a multiple imputation approach to impute missing outcomes. Next, we will use inverse probability weighting with propensity scores. This is a two-step method: 1) using logistic regression, the probability of missingness is modeled as a function of baseline covariates and baseline values of the outcome and 2) the inverse of the propensity scores (predicted probabilities of dropout from the first step) serve as weights in our regression model of the outcomes. Provided the data are missing at random (MAR) or that the probability of missingness can be fully explained by observable data, this approach produces asymptotically unbiased estimates. To allow for the possibility that the MAR assumption may not hold, we will also use a third approach, pattern mixture models, in which the distribution of the outcome is assumed to follow a mixture of two distributions: one for those who complete follow up and another for those who do not.

2.7.3. Sample size and power estimates.

Following the full factorial design, each of the planned 384 participants is randomly assigned to each of the five intervention components. There are 32 possible combinations of intervention components, but each component will be administered to 192 persons and withheld from 192 persons. Using the generic approach described by Lehr,⁶⁴ a sample size of n = 192 per group permits the detection of standardized effect sizes (SEs) of Δ = (4/n^.5) (where 4 encompasses standard assumptions of a type-I and type-II error rate of α = .05 and β = 20%, respectively where 4 ≈ √2 × (|zα/2 + zβ |), and solves to SE = 0.29). According to estimates based on our preliminary data, RxWL without additional intervention components produces ≥5% weight loss for 50% of the sample, and the standard deviation on percent weight change will be 5.8% at 12-months. This generic approach estimates power to detect intervention component effects of 9% on proportion achieving ≥5% weight loss and effects of 1.7% on mean weight loss. Statistical power was further evaluated using Monte Carlo estimation, the model specification (with effect coding) for the proposed study, and parameter estimates based on previous RxWL trials. This tailored approach yielded estimates of 91% power to detect a component marginal effect size of ES = 0.16 or an additional 3% achieving ≥5% weight loss and just under 1% additional weight loss due to component effects. The study is adequately powered because a lesser effect would not be clinically significant.

2.8. Optimization criteria.

The optimization criteria consist of a priori rules for deciding which of the innovative intervention components to include in implementation and dissemination studies of RxWL in the MOST Confirmation Phase.¹⁴ The decision-making approach described by Collins et al. will be used.⁶⁵ In summary, components with significant main effects in the desired direction for proportion of participants achieving a ≥5% weight loss, and/or mean weight loss, will be tentatively selected for inclusion in the treatment package. Interactions will then be considered. Two components that are synergistic (i.e., produce better outcomes together than the additive effect of the two on their own) will both be included. More complex patterns of interactions will then be evaluated individually to determine a final combination of components that produces the best outcomes using the fewest components. The effect of the complexity of the intervention (i.e., number of components assigned) on outcomes will be considered. In the event that a more complex intervention is associated with worse outcomes, components that do not contribute to improved weight loss outcomes will be eliminated. Cost is not a consideration in the optimization criteria because each component is associated with little, if any, additional incremental cost per participant treated.

3. Discussion

Obesity remains an epidemic in the United States and most other developed countries.^66–68 BLI is recommended as a first line treatment, and automated online delivery dramatically improves reach and cost while reducing barriers associated with frequent clinic visits.¹ However, automated online BLI is only initially effective for half of patients or fewer.^8–10 Optimization of automated online BLI thus carries important potential for clinical and public health impact. This project aims to improve both the number of participants who achieve a minimum clinically significant benefit and mean weight loss.

Capitalizing on the MOST framework to guide the research methods is an innovative approach to improving the efficiency and scientific value of the study. The use of a factorial experiment avoids traditional pitfalls of RCTs, which would require a far larger sample to test each intervention component individually and would not be able to disentangle effects of individual intervention components if tested as a combined treatment package. A systematic approach to treatment development and refinement is particularly important for online interventions in which a nearly unlimited number of tools and resources could be provided which risks diluting the impact of the intervention by wasting time and effort on strategies that are not effective. The factorial experiment used in this study will help craft the leanest treatment package most likely to produce the best outcomes. Per the MOST framework, this refined treatment package can then be tested against the preexisting RxWL program in a traditional RCT to more definitively establish the degree to which efficacy has been improved.

The potential of this approach has already been realized once previously in the context of a remotely-delivered weight management program. The OPT-IN study tested telephone coaching calls (12 vs. 24), progress reports to a primary care provider, text messages, meal replacement recommendations, and training of peer support “buddies” as adjuncts to a 6-month mHealth weight management program with lessons and smartphone-based self-monitoring.^69,70 Per optimization criteria that emphasized efficacy and limiting cost to no more than $500 per patient, the study results led to a finalized treatment package that included 12 coaching calls, buddy training, and primary care provider progress reports. An important distinction between OPT-IN and the current study is our emphasis on testing components that can be delivered entirely online without the need for human intervention delivery or additional cost, thus maintaining the fully automated nature of RxWL and the ability to disseminate the program at near zero cost to patients or providers.

Our factorial experiment also provides an important ability to evaluate mechanisms of treatment. Via mediation analysis, this study will test not only which of the experimental components are (or are not) effective, but also why or how they exert their effects. This approach adds value to the research by explaining whether ineffective intervention components did not have the expected effect on weight loss because they had an insufficient effect on their immediate behavioral/psychosocial target(s), and/or because improving the behavioral/psychosocial target did not translate to improved weight loss outcomes.

Per the MOST framework,¹⁴ our study also includes a Preparation Phase project to ensure proper functioning of the experimental intervention components prior to the factorial experiment. The Preparation Phase also frequently involves initial work to establish an effect of the intervention components on their immediate, proximal, intervention targets and/or work to establish levels (e.g., “doses”) of the intervention components to be tested in the Optimization Phase. The Preparation Phase is important as problems such as a lack of acceptability and feasibility can interfere with a robust test of intervention components in the Optimization Phase. The experimental intervention components to be tested in this study are based on well-established behavioral theory and in most cases the approach to be tested has already demonstrated effects on proximal targets. Likewise, there is no emphasis on testing varying “doses” of intervention components; each component to be tested has two binary levels in the factorial experiment corresponding to an on or off state. Therefore, the Preparation Phase in this project is focused on maximizing acceptability, feasibility, and engagement of the experimental intervention components, alone and in combination with each other, prior to the Optimization Phase.

This study is not without limitations. While interaction effects will be considered (e.g., two components are only effective when provided in combination), statistical power for these tests is lower than those for tests of main effects (i.e., the efficacy of each experimental intervention component in isolation). In addition, there is a small risk of contamination between intervention components for participants receiving the platform for social support and friendly competition. As participants in this condition will have the opportunity to communicate with each other, they may share information learned in another component with peers that do not have access to that component. This will be explicitly discouraged but cannot be directly controlled. Lastly, due to the COVID-19 pandemic, study visits for orientation, randomization, and assessment may have to be conducted remotely, for example using wireless scales and self-reported height, which may not be as accurate as measurements made with research-grade equipment. Participant safety must receive the highest priority, with research integrity also receiving a high, but secondary, level of prioritization. In the event that wireless scales are provided to participants for assessment purposes, they will be provided for only as long as necessary to collect an assessment weight, and then returned to the research center. Study staff will guide participants by phone in the protocol for using the scale to provide an assessment weight, which involves placing the scale on a hard surface, weighing in light clothing without shoes, and stepping on and off 3 times to ensure that a reliable measure is obtained.

4. Conclusions

This study will test five experimental intervention components as adjuncts to an existing automated online BLI using a factorial experiment informed by the MOST framework. The primary outcomes are the proportion of participants achieving a minimum clinically significant weight loss of ≥ 5% of initial body weight and mean weight loss. Mediation analysis will be conducted to test hypothesized mechanisms of action and moderators analysis will be conducted to understand for whom and under what circumstances the interventions are effective. Optimization criteria will be used to determine which, if any, of the experimental intervention components should be included in a finalized treatment package.