A factorial experiment to optimize remotely delivered behavioral treatment for obesity: Results of the Opt-IN study

Abstract

Objective:

Intensive behavioral obesity treatments face scalability challenges, but evidence is lacking about which treatment components could be cut back without reducing weight loss. The Opt-IN study applied Multiphase Optimization Strategy (MOST) to develop an entirely remotely delivered, technology-supported weight loss package to maximize the amount of weight loss attainable for ≤$500.

Methods:

Six-month weight loss was examined among adults (N=562) with BMI≥25 who were randomly assigned to conditions in a factorial experiment crossing five dichotomous treatment components set to either low/high (12 versus 24 coaching calls) or off/on (primary care provider [PCP] reports, text messaging, meal replacements, buddy training).

Results:

84.3% of participants completed the final assessment. The treatment package yielding maximum weight loss for ≤$500 included 12 coaching calls, buddy training, and PCP progress reports, produced average weight loss of m=6.1 kg; 57.1% losing ≥5%; 51.8% losing ≥7%; and cost $427/person. The most expensive candidate treatment component (24 coaching calls vs. 12) was screened out of the optimized treatment package because it did not increase weight loss.

Conclusions:

Systematically testing each treatment component’s effect on weight loss made it possible to eliminate more expensive but less impactful components, yielding an optimized resource-efficient obesity treatment for evaluation in an RCT.

Introduction

Practice guidelines advise clinicians to offer or refer adults with obesity to intensive, multicomponent behavioral weight loss treatment¹. The Diabetes Prevention Program (DPP), the gold standard behavioral weight loss treatment, and similar interventions have effectively produced clinically meaningful weight loss and improved health in adults with overweight and obesity^2–5. However, these programs are expensive to deliver (exceeding $1000 in the first year), largely due to the personnel costs for at least 14 professionally-led treatment sessions during the first 6 months⁶. Reducing treatment intensity and implementing the DPP in community or primary care settings lowers annual program cost to an average of $653 per participant, which is still out of reach for many, and decreases intervention effectiveness⁷.

The challenge of increasing the population impact of obesity interventions can be framed as an optimization problem: i.e., to systematically identify the set of intervention components that delivers meaningful weight loss while remaining scalable. However, because behavioral weight loss interventions have traditionally been deployed as bundled “treatment packages,” an evidence base is lacking to guide decisions about which intervention components at which intensity should be included in this set.

The Multiphase Optimization Strategy (MOST)^8,9 framework offers a toolbox of experimental designs that can be used to optimize interventions systematically so they efficiently achieve a stated optimization criterion without resource overuse (e.g. achieve the best clinical outcome possible without exceeding a specified per-participant cost). One design, the factorial experiment, can be conducted to examine effects of individual treatment components and their interactions to determine which components and component levels, singly or combined, make important contributions to the desired outcome. That information, with data on cost, then guides decision-making about assembling an optimized treatment package that best achieves target outcomes within resource constraints. Factorial experiments are often more economical than alternative approaches (e.g., multiple sequential RCTs) because they test multiple component effects simultaneously, requiring fewer participants to achieve the same statistical power¹⁰.

Despite many calls to understand which treatment components are essential to produce meaningful weight loss^6,11–13, few studies have systematically examined this question. The Optimization of Remotely Delivered Intensive Lifestyle Treatment for Obesity (Opt-IN) study¹² aimed to develop an optimized, scalable version of a remotely delivered, technology-supported weight loss intervention by determining which of 5 treatment components contributed most meaningfully and cost-efficiently to weight loss over a 6- month period.

Based upon social cognitive theory^14,15 and prior mHealth obesity intervention trials^16–19, we posited a need for multilevel intervention, testing 2 components aiming to enhance the individual’s weight regulation skills and attitudes and 3 addressing environmental facilitators and barriers of weight loss success²⁰. The 2 individually targeted components, coaching and text messaging, aimed to enhance the person’s self-regulation abilities and self-efficacy about managing diet and physical activity¹⁴. Buddy training and primary care provider (PCP) report aimed to foster a facilitating environment for weight loss by prompting others in the participant’s interpersonal network to convey social support and accountability for weight management¹⁵. The third environmentally targeted component, meal replacements, aimed to overcome a weight regulation barrier by simplifying portion control¹⁵.

The primary aim of Opt-IN was to determine which intervention components maximize weight loss. The secondary aim was to integrate these findings with cost data to build a treatment package producing the greatest weight loss attainable for ≤$500 (selected because the CDC and commercial insurers consider this a reasonable cost to deliver the DPP to an individual).²¹ To our knowledge, this was the first study using MOST to optimize an adult obesity treatment.

Methods

Opt-IN involved a factorial experiment that randomized 562 participants to one of 32 experimental conditions representing all possible combinations of five treatment components. The purpose of the experiment was to estimate the effect of each component, and any component interactions. The primary outcome was weight loss at the end of the 6-month intervention. The study protocol, design, and a corrigendum were published previously^12,22. Participants were recruited throughout the Chicagoland area between 2013 and 2017 by flyers, public transit advertisements, research registries, and word of mouth. Randomization at the rate of approximately 168 participants/year occurred over 40 months.

Procedure

Eligible participants¹² were required to: be 18–60 years old; have a BMI between 25–40 kg/m² (i.e., have overweight or obesity); be weight stable; neither be enrolled in a formal weight loss program nor taking medications known to cause weight loss or gain. They were also required to own an iPhone or Android smartphone, have a PCP, and be able to enlist a weight loss support “Buddy” from their existing social network who was at least 18 years old and had Internet access. Participants were excluded if they had an unstable medical condition (e.g., uncontrolled hypertension), had contraindications to engaging in moderate-vigorous physical activity (MVPA), used insulin, had Crohn’s disease or obstructive sleep apnea, had physician-diagnosed plantar fasciitis, used an assistive device (e.g., cane) for mobility, had been hospitalized recently for psychiatric reasons or expressed current suicidality, were pregnant, lactating, or trying to conceive, met criteria for an eating disorder, endorsed substance abuse or dependence (per DSM-IV criteria), or followed a strict dietary regimen incompatible with study goals. To reduce contamination, participants were also excluded if they lived with another study participant, or had already participated in the study as a “Buddy.”

Participants were screened for eligibility through a multi-step process that included an online web screener, phone screener, and in-person group orientation and equipoise induction session during which study candidates discussed the advantages and disadvantages of different treatment conditions to equalize their desirability prior to randomization.²³ At orientation, participants underwent a written informed consent process and provided contact information and their PCP’s medical approval to participate. Next, an in-person baseline assessment involved additional screening (e.g., depression, BMI) and demographic information (e.g., age, race, ethnicity, gender, socioeconomic status [SES]). SES was self-reported on a scale that ranged from 1 (poor) through 5 (middle class) to 9 (wealthy). Eligible candidates were scheduled for in-person randomization¹², 3-month and 6-month follow-up assessments when a blinded assessor measured weight using a calibrated balance beam scale. Participants received an honorarium of $20 for each follow-up assessment.

A CONSORT diagram showing participant flow through the study appears in Figure 1. Data were collected and maintained using Research Electronic Data Capture (REDCap)^24,25 hosted by Northwestern University. All study protocols were approved by the Northwestern University Institutional Review Board (IRB).

Figure 1:

CONSORT diagram depicting participant flow through the six month study.

Randomization

Participants were recruited in two cohorts. Cohort 1 (n=289) was randomized to conditions 1–16; Cohort 2 (n=273) was randomized to conditions 17–32^12,22. Randomization was stratified by gender and performed in randomly permuted blocks. Interventionists and participants were informed of the randomized assignment. Outcome assessors were blinded to participants’ assigned treatment condition, behavioral adherence, and weight loss trajectory.

Intervention

Each of the 32 experimental conditions (shown in Table 1) included delivery of up to 5 behavioral components, per the factorial design. Regardless of their randomized condition, all participants, also received the CORE intervention described below.

Table 1.

Opt-IN factorial design with 32 conditions^*

Intervention Target	Individual		Environment
Combination	Coaching Calls	Texts	Meal Replacement	PCP Reports	Buddy Training
1	12	No	No	Yes	No
2	12	No	Yes	Yes	Yes
3	12	Yes	No	Yes	Yes
4	12	Yes	Yes	Yes	No
5	12	No	No	No	Yes
6	12	No	Yes	No	No
7	12	Yes	No	No	No
8	12	Yes	Yes	No	Yes
9	24	No	No	Yes	No
10	24	No	Yes	Yes	Yes
11	24	Yes	No	Yes	Yes
12	24	Yes	Yes	Yes	No
13	24	No	No	No	Yes
14	24	No	Yes	No	No
15	24	Yes	No	No	No
16	24	Yes	Yes	No	Yes
17	12	No	No	No	No
18	12	No	Yes	No	Yes
19	12	Yes	No	No	Yes
20	12	Yes	Yes	No	No
21	12	No	No	Yes	Yes
22	12	No	Yes	Yes	No
23	12	Yes	No	Yes	No
24	12	Yes	Yes	Yes	Yes
25	24	No	No	No	No
26	24	No	Yes	No	Yes
27	24	Yes	No	No	Yes
28	24	Yes	Yes	No	No
29	24	No	No	Yes	Yes
30	24	No	Yes	Yes	No
31	24	Yes	No	Yes	No
32	24	Yes	Yes	Yes	Yes

^*Adapted from Pellegrini et al (8, 18)

CORE

The CORE involved a smartphone application (app) showing personalized goals for diet, physical activity, and weight (Figure S1)and online lessons (Figure S2). CORE intervention invoked three behavior change techniques (goal setting, self-monitoring and feedback) that Control Theory posits enhance self-regulation^26,27. Participants downloaded a custom-designed Opt-IN app for self-monitoring all food/drink intake and physical activity (PA) throughout the day, and recording weight daily. They were given a 6-month weight loss goal of 7% of initial body weight, daily calorie and fat gram intake goals based on the DPP², and a weekly PA goal that increased gradually from 100 to 300 minutes of MVPA over the course of the intervention. The app displayed feedback about calorie and fat intakes, MVPA, and weight relative to goals; these data were transmitted to a coach dashboard (Figure S3). Participants received access to online lessons about weight-related topics adapted from the DPP (e.g., basic nutritional information, barriers to PA, setting SMART goals) that they discussed with their coach.

Behavioral Intervention Components (Factors)

Five behavioral intervention components functioned as dichotomous factors set to one of two levels (either low/high or off/on) to which participants were randomly assigned.

Coaching Calls.

Participants received either 12 biweekly or 24 weekly 10–15 minute calls from a health coach, during which the coach discussed CORE lessons and reviewed data transmitted from the participant’s app to the coach dashboard to address self-monitoring adherence, progress toward goals, and motivation to change. Participants retained the same coach throughout the study, except in cases of staff absence or transition.

Progress Report to Primary Care Physician (PCP).

Participants were randomly assigned to have reports about their weight loss progress sent to their PCP or not. Those assigned to PCP report “on” were reminded on every call that their PCP would be mailed a tailored progress report after their 3- and 6-month assessments. The report showed participant’s weight trajectory from baseline onward and provided topical recommendations for PCP-patient discussion.

Text Messages.

Participants were randomly assigned to receive text messages throughout the study or not. Messages were sent as automated push notifications on a schedule determined by: a) the participant’s stated preferences and b) times when the participant was detected to be engaging with the app. Those assigned to receive texts received 7 messages dispersed throughout each week. They could also opt to receive 2 “bonus” messages conveying general weight loss information. Messages addressed pre-specified daily topics and were tailored based on participants’ self-monitored progress toward goals. Based on social cognitive theory,^14,15 message framing conveyed either supportive accountability (e.g., “Awesome job meeting the physical activity goal.” “Remember to log your foods to stay on track”) or facilitation (e.g., “Keeping sliced veggies in your fridge can steer you toward a low-cal snack”).

Meal Replacement (MR) Recommendations.

At randomization, those assigned to MR replacement were given a week’s supply of pre-packaged shakes and bars, and asked to consume these daily to supply part of their calorie and nutrient intakes. After the initial week, those assigned to MR “on” were asked to purchase their own MR supplies because providing MR supplies continually was considered un-scalable., On coaching calls, they were advised to continue using MR products.

“Buddy” Training.

All participants entered the study with a “buddy” of their own choosing to provide support. Participants were randomized either to have their buddy undergo training about how to behave supportively, or not to undergo this training. Buddies assigned to receive training were asked to complete 1 individual coaching phone call and up to 4 online group training webinars. During webinars, a facilitator conveyed skill-building lessons and led peer problem-solving about how to provide effective weight loss support. Buddies earned $5 for each webinar attended, plus a $20 bonus if they attended 3 of the 4 webinars.

Treatment Fidelity

Treatment fidelity was assessed quarterly using a checklist to score a random sampling of 15% of each coach’s telephone coaching calls. The scoring system added points for required treatment elements the coach delivered correctly and subtracted points for contaminating treatment elements the coach delivered from an unassigned condition. If the average fidelity for any coach fell below 90%, retraining occurred.

Average treatment fidelity across all coaches was 99.3% throughout the 5-year study, indicating that treatment components were delivered as intended. None of the 9 coaches required retraining. Coaches assured text messaging receipt by querying about message delivery and detecting and fixing any technical difficulties. Coaching call receipt was verified by audiotape: almost twice as many calls were delivered in the 24- versus the 12-call coaching condition, and total call duration was substantially longer (p<.001) (Table 4). Calls were approximately 2 minutes shorter in the 24- than the 12-session condition (p<.001).

Table 4.

Coaching Call Receipt for 12 versus 24 Call Coaching Conditions

	12-call condition	24-call condition
M (SD) Calls Completed	10.43 (2.65)	19.10 (6.13)***
M (SD) Total Call minutes	159.60 (71.31)	240.13 (121.04)***
M (SD) Minutes Per Call	15.40 (5.61)	12.50 (4.55)***

^***p < .001

Outcomes

Because the primary aim of the Opt-IN study was to determine which intervention components maximize weight loss, the primary outcome was weight loss from baseline to 6 months. As an optimization trial to identify promising components for inclusion in an optimized treatment package⁸, Opt-IN’s criterion to consider an effect important was set liberally at p<.10 to decrease the Type II error rate⁸.^¥

The secondary aim was to apply these results, together with cost data, to build the treatment package producing the greatest weight loss attainable for $500 or less. First we estimated the costs of each component and component level (Table 2) from the perspective of an organization that would implement Opt-IN. We used study records to estimate staff time, including supervision and training, to deliver the core intervention, telephone coaching calls, and buddy training, and to prepare text messages and primary care provider reports. Staff salaries plus 25% fringe benefits were calculated based on median salaries for each of the 3 Bachelors level staff categories (research assistant assessor, project coordinator/coach, programmer) plus the PhD-level clinical supervisor. Equipment and subscription costs included fees to access the study’s web server and the webinar service used to deliver buddy training. Supplies, including printing, paper, and postage for PCP reports, as well as MR shakes and bars were estimated based on actual 2016 prices. The cost of the telephone coaching calls was calculated separately for the 12 call and 24 call conditions. Cost estimates were adjusted from previously published values¹² to reflect real costs to deliver the intervention to 168 participants per year.

Table 2.

Cost per person of lower and higher level of intervention components

Intervention component	Lower level	Higher level
Core intervention	$174	$174
Telephone coaching calls	$150	$276
Primary Care Provider (PCP) reports	$0	$13
Text messages	$0	$26
Meal replacement	$0	$33
Buddy training	$0	$90
TOTAL	$324	$612

Statistical Analysis

Data, analyzed on an intent-to-treat basis using mixed models, allowed for a full unstructured variance-covariance matrix for the repeated measures, and used SPSS Mixed (version 26.0.0.0 64bit) with restricted maximum likelihood estimation to estimate the model parameters. Effect coding was used: off/low component levels were coded as −1, on/high component levels were coded as 1. Cohort was entered into the model as a covariate. Dummy variables for time were included to represent change relative to baseline at the 3- and 6-month follow-ups.

Decision-making proceeded as follows ^8,28. First, all 2-way interactions between component and time were examined (in our longitudinal mixed model this is conceptually equivalent to a component main effect on weight change). Components that showed important interactions with time at the six month primary study endpoint (p<.10) were tentatively selected. Next, all interactions, from lower to higher order, that included important component by time interactions were examined. Tentative decisions made about component inclusion were reconsidered based on important synergistic or antagonistic interaction effects involving initially selected components. Components identified in this manner formed the screened-in set.

With the screened-in set of components identified, decision-making began for the secondary aim. Treatment effect estimates and costs of all possible treatment packages involving these components were examined. Estimated weight loss was calculated based on a parsimonious regression model that included as predictors only the screened-in components, interactions that led to their selection, and corresponding lower order terms. A version of this parsimonious model (excluding interactions with time) was used to estimate the proportion of participants achieving ≥5% and ≥7% 6-month weight loss..

Accounting for attrition led to a sample size estimate of 560 to yield 80% power to detect an effect size of .25 (Cohen’s d) under a two-tailed hypothesis test. Assuming an estimated standard deviation of 4 kg²⁹, this translates to 1 kg difference in weight loss from baseline to 6 months.

Results

Opt-IN participants were mostly female (81.5%), white (74.1%), middle-class ($\overline{X}$ = 5(1.4), and self-identified as non-Hispanic/Latinx (86.8%). Mean BMI at baseline was 32.3(3.6) kg/m². There were no significant differences in sex, race, or age as a function of the component levels to which participants were assigned (Table 3). However, component level and ethnicity interacted (p=0.037), such that fewer self-identified Hispanic/Latinx and Other-identified ethnicities than non-minorities were randomized to receive 24 rather than 12 sessions of coaching (p=.007).

Table 3.

Baseline participant demographics by component level^*

Factor Level	Sex		Race			Ethnicity			Age
	Male	Female	White	Black	Other	Not Hisp	Hispanic	Other	Mn (SD)
	(n=103)	(n=459)	(n=417)	(n=87)	(n=58)	(n=489)	(n=53)	(n=20)	38.9(10.9)
Coaching
12	51(49.5)	229(49.9)	206(49.4)	40(46.0)	34(58.6)	232(47.4)	33(62.3)	15(75.0)	38.8(11.1)
24	52(50.5)	230(50.1)	211(50.6)	47(54.0)	24(41.4)	257(52.6)	20(37.7)	5(25.0)	39.1(10.8)
PCP
No	51(49.5)	229(49.9)	211(50.6)	40(46.0)	29(50.0)	244(49.9)	26(49.1)	10(50.0)	38.1(11.0)
Yes	52(50.5)	230(50.1)	206(49.4)	47(54.0)	29(50.0)	245(50.1)	27(50.9)	10(50.0)	39.7(10.8)
Texts
No	50(48.5)	231(50.3)	213(51.1)	38(43.7)	30(51.7)	243(49.7)	32(60.4)	6(30.0)	39.0(10.4)
Yes	53(51.5)	228(49.7)	204(48.9)	49(56.3)	28(48.3)	246(50.3)	21(39.6)	14(70.0)	38.9(11.4)
Meal
No	51(49.5)	231(50.3)	205(49.2)	46(52.9)	28(53.4)	240(49.1)	32(60.4)	10(50.0)	38.5(10.6)
Yes	52(50.5)	228(49.7)	212(50.8)	41(47.1)	31(46.6)	249(50.9)	21(39.6)	10(50.0)	39.4(11.2)
Buddy
No	50(48.5)	230(50.1)	206(49.4)	39(44.8)	35(60.3)	239(48.9)	30(56.6)	11(55.0)	39.0(10.7)
Yes	53(51.5)	229(49.9)	211(50.6)	48(55.2)	23(39.7)	250(51.1)	23(43.4)	9(45.0)	38.9(11.1)
Test (p)	x2=0.22 (p=.999)		x2=8.82 (p=.549)			x2=19.31 (p=.037)			F=.697 (p=.626)

^*Sex/Race/Ethnicity: Count (% by Factor Level); Age: Mean (Standard Deviation). Omnibus tests of distribution across levels of all factors (Gender: Logistic Regression; Race/Ethnicity: Nominal Regression; Age: ANOVA)

Of the full sample of 562 individuals who began the study, 474 (84.3%) completed the 6-month endpoint, 64 (11.4%) were lost to follow-up, and 24 (4.3%) formally withdrew. None of these showed a differential effect of treatment component level.

Table 5 shows abbreviated results of the full model testing all main effects and interactions of treatment components on the primary 6-month weight loss outcome. Effects on 3-month outcomes are omitted for brevity, but all effects from the full model appear in Table S1. Table S2 shows 6-month weight loss for both levels of each intervention component. Mixed model results (Buddy×Time interaction) showed that buddy training was the only component whose inclusion significantly increased 6-month weight loss above the effect of CORE. Therefore, we tentatively included buddy training in the screened-in set of components and excluded the other four components. Next, we reconsidered these decisions in the light of important interaction effects. We particularly examined interactions with Buddy to determine whether any of the four omitted components should be included in the screened-in set, even though they did not show two-way interactions with time, because they boosted or reduced the effect of buddy training.

Table 5:

Abbreviated full mixed model examining effects on 6-month weight loss^*

Effect	Estimate	t	p	95% Confidence Interval
Baseline
Intercept	91.093	114.382	0.000	89.529, 92.658
Cohort	−2.170	−1.899	0.058	−4.414, 0.075
Coaching0	−0.106	−0.185	0.853	−1.227, 1.016
PCP0	0.796	1.393	0.164	−0.326, 1.917
Text0	0.242	0.423	0.672	−0.880, 1.363
Meal0	−1.114	−1.951	0.052	−2.236, 0.008
Buddy0	0.464	0.813	0.417	−0.658, 1.586
3-Months
Time	−3.761	−25.237	0.000	−4.054, −3.468
6-Months
Time	−4.841	−22.284	0.000	−5.268, −4.415
Time×Coaching	0.124	0.572	0.567	−0.303, 0.551
Time×PCP	−0.023	−0.104	0.917	−0.449, 0.404
Time×Text	0.084	0.387	0.699	−0.343, 0.511
Time×Meal	0.108	0.498	0.619	−0.319, 0.535
Time×Buddy	−0.435	−2.003	0.046	−0.862, −0.008
Time×Coaching×PCP	−0.130	−0.601	0.548	−0.556, 0.296
Time×Coaching×Text	−0.179	−0.824	0.411	−0.604, 0.247
Time×Coaching×Meal	−0.096	−0.444	0.657	−0.522, 0.330
Time×Coaching×Buddy	−0.085	−0.394	0.694	−0.511, 0.341
Time×PCP×Text	−0.211	−0.974	0.331	−0.637, 0.215
Time×PCP×Meal	0.093	0.429	0.668	−0.333, 0.519
Time×PCP×Buddy	−0.276	−1.273	0.204	−0.702, 0.150
Time×Text×Meal	0.101	0.465	0.642	−0.325, 0.527
Time×Text×Buddy	0.217	1.003	0.316	−0.208, 0.643
Time×Meal×Buddy	0.041	0.188	0.851	−0.385, 0.467
Time×Coaching×PCP×Text	−0.074	−0.342	0.733	−0.500, 0.352
Time×Coaching×PCP×Meal	0.362	1.6700	0.096	−0.064, 0.788
Time×Coaching×PCP×Buddy	0.090	0.413	0.680	−0.336, 0.515
Time×Coaching×Text×Meal	−0.062	−0.286	0.775	−0.488, 0.364
Time×Coaching×Text×Buddy	0.068	0.313	0.755	−0.358, 0.494
Time×Coaching×Meal×Buddy	0.008	0.035	0.972	−0.418, 0.434
Time×PCP×Text×Meal	−0.165	−0.760	0.448	−0.591, 0.261
Time×PCP×Text×Buddy	0.424	1.956	0.051	−0.002, 0.850
Time×PCP×Meal×Buddy	0.112	0.517	0.606	−0.314, 0.538
Time×Text×Meal×Buddy	−0.186	−0.856	0.392	−0.612, 0.240
Time×Coaching×PCP×Text×Meal	0.204	0.941	0.347	−0.222, 0.630
Time×Coaching×PCP×Text×Buddy	−0.383	−1.769	0.078	−0.809, 0.042
Time×Coaching×PCP×Meal×Buddy	−0.376	−1.736	0.083	−0.802, 0.050
Time×Coaching×Text×Meal×Buddy	−0.171	−0.790	0.430	−0.597, 0.255
Time×PCP×Text×Meal×Buddy	−0.027	−0.123	0.902	−0.454, 0.400
Time×Coaching×PCP×Text×Meal×Buddy	−0.164	−0.755	0.451	−0.590, 0.262

⁰Main effects from coaching to buddy represent component effects at baseline (time zero) and therefore do not test the hypotheses of interest.

^*Effects that meet the criterion of importance (interacting with Time or with a screened in component and p<.10 at 6 months) are designated in bold. The full analytic model from which these estimates are derived also contains effects and interactions at time 3-months, corresponding to the 6-month effects that are of primary interest and shown here. For brevity, 3-month outcomes have been omitted from this table but are presented in Table S1.

No other 2- or 3-way interactions were identified as important; however, two 4-way and two 5-way interactions met the p<.10 criteria for importance. The 4-way interaction of Time×Coaching×PCP×Meal was not considered further because it did not involve Buddy. The interaction of Time×PCP×Text×Buddy was identified as important, so we examined a plot of the interaction (Figure 2), to determine whether components were synergistic or antagonistic with Buddy. As shown, the greatest weight loss occurred when Text was off and both Buddy and PCP were on. Therefore, we preliminarily considered including both Buddy training and PCP reports in our screened in set. The two remaining important effects were 5-way interactions involving Time×PCP×Buddy×Coaching×Meal and Time×PCP×Buddy×Coaching×Text. Examination of the first interaction (Figure S4) showed a reduced effect of Meal when Buddy and PCP were included components, so we continued to screen out meal replacement recommendations. Examination of the second of these interactions (Figure S5) continued to support the decision to leave PCP reports in and text messages out of the screened-in set. Neither plot showed an advantage for the higher level of coaching calls when Buddy was turned on, so we left coaching at its lower, less expensive level.

Figure 2:

Effect of Buddy×PCP×Text interaction on weight change (kg) at six months. Plots on the left (A) show weight gain or loss among those who received buddy training, with text messaging and PCP report turned on or off. Plots on the right (B) show weight change among those who received no buddy training with text messaging and PCP report either on or off. Error bars depict the 95% confidence interval of each estimated mean. Abbreviations: BUDDY=buddy training; PCP=progress report sent to primary care provider; TEXT=text messaging.

For our secondary aim of identifying the combination of components that produces the maximum expected weight loss attainable for <$500 (i.e. the optimized intervention), we computed the average expected weight loss and expected proportion achieving 5% and 7% weight loss for each combination of components in the screened-in set based on parsimonious regression models. These quantities and estimated cost appear in Table 6. Combination 21, which costs $427, adds the synergistic effect of PCP report to buddy training, achieving an estimated average 6-month weight loss of 6.1 kg, with an expected 57.1% of the sample losing 5% and 51.8% losing 7% of their initial body weight.

Table 6:

Predicted 6-month weight change and cost of candidate component combinations (i.e., intervention packages)

Combination						Estimated Effects			Cost
#	Calls	Text Message	Meal Replacement	PCP Report	Buddy Training	6-mo weight change (kg)	% achieving 5% wt loss	% achieving 7% wt loss
21	12	No	No	Yes	Yes	−6.1112	57.13	51.77	$427
1	12	No	No	Yes	No	−3.3966	34.48	25.86	$337
5	12	No	No	No	Yes	−5.0540	46.56	31.02	$414
17	12	No	No	No	No	−5.2389	52.95	41.17	$324

Discussion

The Opt-IN study addressed barriers to the institutional uptake of remotely delivered intensive lifestyle interventions for obesity by applying the MOST framework. We performed a factorial optimization trial designed to determine which of five components of a remotely delivered, technology-supported obesity intervention contributed importantly to 6-month weight loss in a sample of adults with obesity. That information, combined with data on cost, was then used to explore which combinations of components and component levels best met our pre-specified optimization criterion by producing the maximum weight loss attainable for ≤$500/person. The results suggested that the combination of CORE (i.e., app, goals, online lessons), 12 coaching calls, buddy training, and PCP reports best met our optimization criterion.

It is important to note that the optimization criterion we applied (i.e., maximum weight loss for ≤$500) is not the only one that might be considered. Our criterion gave highest priority to maximizing weight loss, in part because we were able to assume access to resources of at least $500 per treated person, supplied by either insurance or the individual. A different context characterized by greater resource constraints, might call for a different optimization criterion. For example, a program for low-income, uninsured adults might prioritize maximizing population reach, i.e., achieving allocation of scarce financial resources to treat the maximum possible number of people who need obesity intervention³⁰. Here decision-makers might find it an acceptable tradeoff to achieve somewhat less weight loss so that funds could be stretched to benefit a larger population, leading to selection of combination 17 in Table 6. Combination 17, which leaves all components off or to the lowest level, is the least expensive treatment package ($324, versus $427 for combination 21), while still achieving an estimated 5.2 kg. average weight loss, or at least 5% weight loss for more than 50% of participants. Hence, combination 17 offers better value ($61.8/kg lost) than combination 21 ($69.9/kg lost), although it results in somewhat less weight loss.

Of the treatment components we tested, only training the participant’s buddy to be supportive increased average 6-month weight loss, a benefit marginally augmented by adding PCP reports. There was no evidence to suggest that increasing the number of coaching calls from 12 to 24, adding meal replacement recommendations, text messages, or PCP progress reports increased weight loss on its own. The absence of a dosage effect due to doubling the number of coaching calls was unexpected, since prior research has suggested that a greater number of treatment sessions is associated with greater weight loss^31–33. Our analyses of call receipt show that the lack of a dosage effect cannot be attributed to failure of treatment implementation. At least one prior study also found that offering moderate intensity obesity treatment with sessions every other week yielded weight loss results comparable to higher intensity, more frequent treatment, and was more cost-effective³⁴.

The failure to find increasing weight loss benefit as a result of adding more intensive or expensive treatment components is good news for public health. Findings suggest that the average adult who seeks treatment for obesity can lose weight with a relatively low-cost intervention^8,17,35,36. As noted, our results indicate that several different treatment packages produced meaningful weight loss for <$500. The one yielding the greatest weight loss added buddy training plus PCP reports to 12 connected coaching sessions plus the CORE. We interpret this finding to mean that the 12 individually targeted coaching sessions saturated participants’ skill building needs, but that environmentally targeted components addressed otherwise unmet needs.

Conceivably, an even lower cost intervention involving the CORE alone might produce weight loss since the app and online lessons incorporate well-studied behavior change techniques (goal-setting, self-monitoring, feedback) that have been shown in some³⁷ but not all³⁸ systematic reviews to improve diet, physical activity, and weight loss. Although we cannot presently determine how much of the weight loss produced by the optimized treatment package was attributable to CORE versus the added intervention components, we are conducting a trial that, in part, compares the impact on weight loss of obesity treatment that uses the app alone versus app plus coaching³⁹.

This research has limitations, including that it did not test all possible intervention components, evaluated a diluted version of meal replacement for the sake of scalability, and examined weight loss initiation but not maintenance. Very importantly, the weight loss and cost figures we report for different combinations of components are estimates. Further, the developed intervention packages were optimized for adults with obesity who enrolled in a remotely delivered treatment and were predominately non-Hispanic, middle class, white females. Results may not generalize to other subpopulations. Definitive demonstration of the effectiveness of a developed, optimized treatment package requires a test against a comparator in an RCT. If an optimized obesity treatment package demonstrates effectiveness, it can be considered an evidence-based practice (or empirically supported treatment) that has good odds of producing meaningful weight loss for an average adult⁴⁰. Per precision behavioral medicine, however, individual and temporal variability in the response to any single treatment are likely. To address those issues, the MOST research design toolkit includes methods to develop treatment algorithms that can adapt guide the evidence-based practice process in a resource efficient manner^8,9,39–42.

Conclusion

The Opt-IN study demonstrates how a factorial experiment can be used to gather data about which components of an intensive behavioral obesity treatment program contribute meaningfully to weight loss and at what cost. When five components were evaluated for effects on 6-month weight loss in a remotely delivered, technology-supported obesity treatment, training participants’ buddies to be supportive augmented weight loss, a benefit enhanced by providing progress reports to the PCP. Providing 24 rather than 12 coaching calls, meal replacement recommendations, and text messaging produced no additional weight loss. The resulting data were used systematically to guide decisions about which components were essential to achieve meaningful weight loss, which could be reduced or eliminated to reduce cost and burden, and which provided a desired balance of effectiveness and economy. Results suggest an optimally cost-efficient obesity treatment package to maximize weight loss, and an alternative to consider when the context requires reducing treatment cost to extend treatment to a greater number of people.

STUDY IMPORTANCE.

What is already known about this subject? Please remember to also include this between the title page and structured abstract in your paper.

Effective behavioral treatment packages for obesity impose burden and cost that impede scalability, but evidence is lacking about which components could be reduced or eliminated without losing effectiveness

What are the new findings in your manuscript? Please remember to also include between the title page and structured abstract in your paper.

A factorial optimization trial was conducted using Multiphase Optimization Strategy (MOST) to identify a set of intervention components that cost-efficiently enhanced weight loss. When added to a CORE intervention involving an app, goals, and online lessons, an optimized treatment package consisting of 12 health coaching calls, progress reports sent to a primary care physician, and training a support Buddy maximized weight loss at a cost of $427/person. More expensive components (e.g., 24 versus 12 coaching calls) were omitted because they proved cost-inefficient

How might your results change the direction of research or the focus of clinical practice? Please remember to also include between the title page and structured abstract in your paper.

Applying an optimization strategy allows data to be used systematically to guide decisions about which treatment components are essential and which can be reduced or eliminated to reduce cost and burden. This makes it possible to assemble a treatment package that provides a desired balance of effectiveness and economy.