Engagement and outcomes in a digital Diabetes Prevention Program: 3-year update

Abstract

Objective

Translations of the Diabetes Prevention Program (DPP) have proliferated in recent years, with increasing expansion to digital formats. Although these DPP translations have consistently shown favorable clinical outcomes, long-term data for digital formats are limited. This study’s objective was to examine clinical outcomes up to 3 years post-baseline and the relationship between program engagement and clinical outcomes in a digital DPP.

Research design and methods

In a single-arm, non-randomized trial, 220 patients previously diagnosed with prediabetes were enrolled in the Omada Health Program, a commercially available, 16-week DPP-based weight loss intervention followed by an ongoing weight maintenance intervention. Changes in body weight and A1c were assessed annually. Relationships between program engagement during the first year and clinical outcomes across 3 years were examined.

Results

Participants were socioeconomically diverse (62% women, 50.2% non-Hispanic white, 51.7% college educated or higher). From baseline to 3 years, those participants who completed four or more lessons and nine or more lessons achieved significant sustained weight loss (–3.0% and –2.9%, respectively) and an absolute reduction in A1c (–0.31 and –0.33, respectively) with an average remission from the prediabetes range to the normal glycemic range. Factor analysis of engagement metrics during the first year revealed two underlying dimensions, one comprising lesson completion and health behavior tracking consistency, and the other comprising website logins and group participation. When these two factors were used to predict weight loss, only the logins and group participation factor was a significant predictor of weight loss at 16 weeks and 1 year.

Conclusions

This study demonstrates significant long-term reductions in body weight and A1c in a digital DPP and identifies patterns of program engagement that predict weight loss.

Significance of the study.

What is already known about this subject?

• Previous research has shown that digital Diabetes Prevention Programs can produce clinically meaningful outcomes at 1 year. However, there remains a need to validate their long-term effectiveness and elucidate the relationship between program engagement and clinical outcomes.

What are the new findings?

• Weight loss and A1c reduction were significantly maintained from baseline to 3 years; regression analyses of program engagement revealed that website logins and group participation were significantly associated with weight loss at 16 weeks and 1 year.

How might these results change the focus of research or clinical practice?

• This research validates the long-term efficacy of a digital DPP, supporting its expanded use in standard clinical practice, given the potential for national reimbursement through the Medicare Diabetes Prevention Act.

Introduction

Prediabetes and diabetes rates have grown at an alarming pace. In the USA alone, an estimated 86 million individuals (1 in 3) have prediabetes, and 29.1 million (1 in 11) have diabetes.¹ Persons with prediabetes, the clinical precursor to diabetes characterized by elevated blood sugar levels, are estimated to convert to type 2 diabetes at a rate of 5%–10% per year.² However, with early intervention through lifestyle modification (ie, improving nutrition and physical activity habits to achieve modest weight loss), conversion rates are substantially reduced. The Diabetes Prevention Program (DPP) clinical trial demonstrated a 58% reduction in diabetes risk in the lifestyle modification condition relative to placebo over 3 years³ and long-term cost savings and sustained risk reduction.^{4 5} As a result of the success of the DPP, the Centers for Disease Control (CDC) established the National Diabetes Prevention Program,⁶ spawning dozens of successful translations and widespread dissemination of the DPP lifestyle intervention. Subsequent systematic reviews^7–9 and meta-analyses^10–13 have shown that behavior-based lifestyle interventions for diabetes prevention yield significant improvements in weight and glycemic outcomes at 1 year. Only a handful of in-person^14–17 or telephonic^{18 19} translations have examined outcomes at 2 years or beyond; only our previous report²⁰ has done the same with respect to a digital translation.

The CDC has established the Diabetes Prevention Recognition Program (DPRP) standards⁶ to accredit DPP translations that deliver an approved curriculum, provide health coaching and group support, and equip participants with skills and self-monitoring tools to support behavior change. The fundamental components of the intervention are based on social cognitive theory, the theory of planned behavior, and the transtheoretical model and focus on the role of vicarious learning, social support, and goal-setting appropriate to the patient’s stage of change.^{7 21} As of August 30, 2017, the DPRP registry (https://nccd.cdc.gov/DDT_DPRP/Registry.aspx)contains 1500 organizations, of which 136 deliver some or all program aspects digitally.

Although less prevalent than their in-person counterparts, digital DPPs have emerged as a scalable method to reach larger numbers of the at-risk population. Digital DPPs successfully re-create and innovate on the in-person experience by delivering educational curriculum and providing access to a health coach and a supportive group environment, while freeing participants from the requirement of traveling to a specific location or scheduling a specific day and time to participate.

A number of published digital DPP translations have examined associations between one or more aspects of program engagement during the intervention and weight loss at the end of the intervention. Increased weight loss is associated with more frequent lesson completion,^{22 23} body weight tracking,^{24 25} steps tracking,²⁵ interactions with the health coach,²⁵ and logins to the intervention’s website.^{26 27} Importantly, however, these studies have all examined isolated associations between engagement and weight loss (eg, simple correlations). Only a handful of DPP translations have used statistical techniques (eg, multiple regression) that take patterns of association among engagement metrics into account when predicting weight loss, enabling a clearer picture of the ‘unique’ role played by specific facets of engagement.^28–32

The present study adds to the growing literature of digital DPP translations in two ways. First, it reports on two key clinical indicators (body weight and A1c) at the furthest post-baseline time point yet described: 3 years, extending our previous progress reports at 1 year³³ and 2 years.²⁰ Second, it examines patterns of relationship among engagement metrics (using factor analysis) and how those underlying factors predict weight loss (using multiple regression) at various time points (16 weeks, 1 year, 2 years, and 3 years) to elucidate this topic.

Research design and methods

Research design

This study was a quasi-experimental, single-arm, non-randomized longitudinal trial. The 1-year, 2-year, and 3-year follow-up measurements were not originally planned time points in the research design. To increase participation at these time points, various incentive programs were used whereby participants who submitted weight data and an A1c sample received a $20 gift card or a sweepstakes to receive a $50 gift card. The full rules and disclosures of the sweepstakes were sent to all participants prior to their annual anniversaries. The research protocol was approved for exemption by the Western Institutional Review Board for the analysis of de-identified data.

Participants

Participants were recruited through online advertisements. Potential participants were screened for the following eligibility criteria, based on CDC DPRP standards³⁴ : 18 years of age or older at the time of enrollment, body mass index (BMI) of > 24 kg/m² (> 22 kg/m² if the participant self-identified as Asian), able to engage in light physical activity, and diagnosed with prediabetes within the year prior to enrollment. Eligible participants provided informed consent, registered for an online account, completed health and demographic questions, and were enrolled in the intervention.

Program

A full description of the program has been previously published³³ and is briefly summarized here. The Omada Health Program (formerly Prevent) is a digital translation of the DPP lifestyle intervention³ that is accessible via internet-enabled desktop or mobile devices. The program consisted of (1) one year of a behavior change curriculum³⁵ approved by the CDC DPRP; (2) technology-enabled tools to track nutritional intake, physical activity, and body weight; (3) personalized health coaching; and (4) small group support.

Upon enrollment, participants were matched into small peer groups of 10–15 and assigned a health coach. Through a private online social network with asynchronous messaging and facilitated by the health coach, group members discussed their goals and progress and provided social support and advice. One-on-one communication between each participant and his or her health coach was available via telephone, e-mail, and web-based private messaging; coaches were provided with a weekly structured protocol to engage participants and guidance on how best to provide personalized guidance and support.

At any convenient time or place using internet-enabled devices (eg, laptop, tablet, or smartphone), program participants could complete weekly curriculum lessons on lifestyle and behavior change, communicate with the health coach and/or peer group, self-monitor diet and physical activity, and view progress toward their weight loss target (for an illustrative video, visit https://www.omadahealth.com/see-how-it-works). For the duration of the study, participants maintained ad libitum access to the curriculum, tracking tools, and group support, as well as the ability to initiate contact with coaches.

Measures

Baseline demographic and health information were collected at study enrollment with an Internet-based self-report questionnaire.

Weight and BMI

Participants were shipped a wireless weight scale (BodyTrace, New York, NY; https://www.bodytrace.com/medical/) that was linked to the participant’s online account. The scale automatically transmits body weights to Omada’s internal database using the cellular GSM network. The last weight captured prior to the start of the first week’s curriculum was used as that participant’s baseline weight. Participants were instructed at the start of the program to weigh themselves weekly and were reminded via e-mail and telephone calls to weigh in at the 1-year, 2-year, and 3-year assessment time points. Weight measurements were highly stable; the scale’s coefficient of variation was ±0.2 lb.³⁶ Height was collected through self-report at the time the participant set up their online account. BMI (kg/m²) was calculated using baseline weight and height.

Glucose control

Glycosylated hemoglobin (A1c) was measured in percentage units (National Glycohemoglobin Standardization Program/Diabetes Control and Complications Trial) using self-administered AccuBase A1c test kits (DTI laboratories, Thomasville, Georgia, USA; http://www.dtilaboratories.com/accubase-a1c-test-kit.html). The kit includes an FDA-cleared, whole blood finger-stick test that uses a capillary tube blood collection method. Abstaining from food or drink prior to blood collection is not necessary for an A1c test (unlike a fasting plasma glucose test), making it ideal for home-based collection. Kits were mailed to participants at baseline and at 0.5, 1, 2, and 3 years. Participants completed the test, mailed the preserved blood samples to a central processing lab for analysis, and were notified of their results.

Program engagement

The program’s software platform captured multiple points of engagement: logins on the website, completion of weekly curriculum lessons, interactions with the health coach and the group, use of diet and activity tracking tools, and weigh-ins on the wireless scale. Although participants retained access to the digital program throughout the second and third years of the study, engagement was substantially reduced because of the lack of continuing curriculum and active coaching. Thus, the present analysis examined program engagement during three ‘windows’ within the first year only, as follows. Weeks 1–16 were the ‘core phase,’ during which foundational aspects of the intervention with respect to achieving healthy weight loss were introduced, as prescribed in the first set of 16 DPRP lesson modules³⁵). Weeks 17–52 were the ‘sustain phase,’ during which participants focused on reinforcing lifestyle changes and habits critical to successfully maintaining weight loss, as prescribed in the second set of 15 DPRP lesson modules.³⁵ Weeks 1–52 comprised cumulative utilization of program elements.

The following metrics were computed during each time window: (1) ‘Lessons Completed’ was the number of lessons completed; (2) ‘Weight Tracked’ was the number of weeks during which a participant weighed-in at least once; (3) ‘Steps Tracked’ was the number of weeks during which a participant logged their daily walking steps at least once; (4) ‘Group Conversations’ was the number of comments made or replied to on the group board; (5) ‘Group Posts Liked’ was the number of group board comments a participant ‘hearted’ (similar to ‘liking’ content on social media); (6) ‘Login Sessions’ was the number of unique login sessions (desktop or mobile) to the Omada web portal; and (7) ‘Coach Conversations’ was the number of private messages sent from a participant to his or her health coach. This metric was only computed during weeks 1–16, as coaches did not actively reach out to individual participants during weeks 17–52. Tracking of diet was highly variable within and between participants, precluding it from analysis here.

Statistical analysis

Understanding outcomes: linear mixed models

Linear mixed-effects models (LMMs) were used to obtain adjusted mean changes in weight and A1c over the 3-year follow-up period (adjusting for participants’ baseline age and gender), as in our previous papers^{20 33} and other DPP translations.^{16 37 38} An LMM is a likelihood-based model that analyzes all available data and provides unbiased estimates for the model parameters under the assumption that data are missing at random (ie, missingness is independent of the unobserved outcomes conditional on observed data). Rather than deleting any participant who does not have a complete set of outcomes, an LMM incorporates all observed repeated measures of each participant into the likelihood estimate. In this sense, an LMM is superior to a last observation carried forward approach, which requires the assumption of missing completely at random (ie, missingness is independent of both unobserved and observed data) that is rarely met in real data sets.

Based on exploratory data analysis and graphs of the time trends, piecewise linear models were fit for weight change, with days from baseline and a change point at the day of the last Core lesson included in the model. For A1c, an additional change point at 1 year was added to the LMM. Weight change was estimated as a percentage change from baseline weight and A1c change as the raw change (in percentage units) from baseline A1c. All analyses were performed using SAS 9.4 (SAS Institute).

Understanding engagement: correlations, composite scores, and factor analyses

Metrics of participant engagement are expected to be associated for two reasons. First, because engagement with program features is driven by underlying inter-individual differences in motivation, ability, and self-efficacy (ie, individuals with greater self-efficacy would be predicted to engage with the entire program more than individuals with lower self-efficacy). Second, because engaging with some aspects of the program is directly conditional on engaging with other aspects of the program (eg, completing a lesson or messaging the coach requires that a participant first log in). To better understand underlying patterns of engagement and their relationship with weight change, three different statistical techniques were used.

As a primary analysis, exploratory factor analysis with varimax rotation (the PROC FACTOR function in SAS software) was used with the aim of reducing the dimensionality of engagement elements. The resulting factors were then included as independent variables in subsequent multiple linear regression analyses to investigate their ability to predict LMM-adjusted weight change at 16 weeks, 1 year, 2 years, and 3 years.

Two secondary analyses are detailed in the online supplementary file. The first examined simple correlations between engagement metrics and weight change, as is common in the DPP translation literature. The second explored differences in a composite engagement score among participants who were ‘successful’ (ie, weight loss ≥5%), ‘unsuccessful’ (ie, weight loss <5%), or ‘non-reporters’ (ie, those who failed to weigh in) at each time point.

Results

As previously reported,^{20 33} 254 eligible individuals responded to the online advertisements and 220 enrolled in the program. Figure 1 visualizes participant retention throughout the entire study, highlighting the number of participants who (1) provided at least one body weight during the target weigh-in time points: 16 weeks±1 week, 1 year±1 month, 2 years±2 months, and 3 years±4 months; (2) returned an A1c kit at baseline, 6 months, 1 year, 2 years, and 3 years; and (3) met two key CDC-defined thresholds³⁴ related to lesson module completion during the primary phase of the intervention: at least four sessions/lessons and at least nine sessions/lessons (out of a possible 16); these subsamples are sometimes referred to as ‘Starters’ and ‘Completers,’ respectively.20 33 39–41 Because of the relatively low incentive to participate in later analysis time points (ie, a gift card), participant attrition increased over time for both body weight and A1c data (making the use of linear mixed models essential rather than optional).

Figure 1.

Participant retention flow chart. DPRP, Diabetes Prevention Recognition Program.

Table 1 summarizes baseline biometrics and demographics of the enrolled study sample, as well as key analytic subsamples discussed in the paper. On the whole, baseline biometrics and demographics of these subsamples were very similar to each other and to the full sample.

Table 1.

Baseline demographic and biometric characteristics of the full study sample and key subsamples

		Subsamples: by weigh-in window				Subsamples: by lesson completion
	Enrolled (n=220)	16 weeks (n=147)	1 year (n=161)	2 years (n=145)	3 years (n=102)	4+ (n=187)	9+ (n=155)
	M±SD	M±SD	M±SD	M±SD	M±SD	M±SD	M±SD
Age	43.6±12.4	46.0±12.8	44.8±12.2	44.9±12.3	46.4±12.6	43.9±12.4	44.9±12.8
Weight	223.1±47.9	218.7±46.3	221.0±48.4	219.4±47.4	217.9±47.2	222.5±47.0	221.0±47.2
BMI	36.6±7.5	35.8±7.1	36.3±7.8	36.1±7.5	35.9±7.2	36.7±7.6	36.4±7.6
A1c	5.97 ±. 91	6.01 ±. 97	6.01 ±. 97	6.02 ±. 96	6.04±1.02	5.97 ±. 92	5.98 ±. 96
	%	%	%	%	%	%	%
Sex, female	82.7	81.6	82.0	82.1	84.3	85.0	83.9
Ethnicity
White	49.1	53.1	47.8	48.3	49.0	49.7	50.3
Black	28.6	23.1	29.2	27.6	28.4	28.3	27.1
Hispanic	10.5	12.3	11.2	11.7	10.8	11.3	11.6
Other	9.5	9.5	9.9	10.3	9.8	8.0	8.4
Not disclosed	2.3	2.0	1.9	2.1	2.0	2.7	2.6
Education
≥College degree	35.0	39.5	40.4	37.3	39.2	39.0	41.3
<College degree	33.6	29.9	31.1	31.0	29.4	32.1	30.3
Not disclosed	31.4	30.6	28.5	31.7	31.4	28.9	28.4
Annual income
≥$50 000	33.6	36.1	34.2	33.1	34.3	34.8	36.1
<$50 000	31.4	29.3	32.9	32.4	30.4	32.6	32.3
Not disclosed	35.0	34.6	32.9	34.5	35.3	32.6	31.6

BMI, body mass index.

Clinical outcomes over 3 years

Our previous papers^{20 33} reported significant reductions in weight and A1c from baseline at the 16-week, 1-year, and 2-year time points. Table 1 summarizes the linear mixed models extended to the 3-year time point, which were run separately on starters (table 2A) and completers (table 2B). For both subsamples of participants, weight and A1c remained significantly reduced from baseline at the 3-year time point.

Table 2A.

Changes from baseline in body weight and A1c for participants who completed four or more lessons (n=187)

Time point	Weight change (lb)		Weight change (%)		A1c change
	Mean (SE)*	p Value	Mean (SE)*	p Value	Mean (SE)*	p Value
16 weeks	−11.1 (0.7)	<0.0001	−5.0 (0.3)	<0.0001	+0.03 (.06)	0.55
1 year	−10.0 (0.8)	<0.0001	−4.7 (0.4)	<0.0001	−0.38 (.07)	<0.0001
2 years	−8.3 (1.4)	<0.0001	−4.2 (0.8)	<0.0001	−0.43 (.08)	<0.0001
3 years	−6.7 (2.0)	0.0009	−3.0 (0.9)	0.0009	−0.31 (.09)	0.0008

*Adjusted mean and SE values from linear mixed models. At baseline, these participants had an adjusted mean (SE) weight of 221.4 (3.5) lb and an adjusted mean (SE) A1c of 5.99 (0.08).

Table 2B.

Changes from baseline in body weight and A1c for participants who completed nine or more lessons (n = 155)

Time point	Weight change (lb)		Weight change (%)		A1c change
	Mean (SE)*	p Value	Mean (SE)*	p Value	Mean (SE)*	p Value
16 weeks	−11.6 (0.7)	<0.0001	−5.2 (0.3)	<0.0001	+0.03 (.06)	0.62
1 year	−10.2 (0.9)	<0.0001	−4.9 (0.5)	<0.0001	−0.40 (.07)	<0.0001
2 years	−8.3 (1.4)	<0.0001	−4.3 (0.8)	<0.0001	−0.46 (.08)	<0.0001
3 years	−6.3 (2.1)	0.0024	−2.9 (1.0)	0.0024	−0.33 (.09)	0.0005

*Adjusted mean and SE values from linear mixed models. At baseline, these participants had an adjusted mean (SE) weight of 219.8 (3.9) lb and an adjusted mean (SE) A1c of 6.02 (0.08).

Factor analysis of engagement metrics

Examination of eigenvalues and scree plots from the factor analysis of engagement metrics within the windows weeks 1–16, weeks 17–52, and weeks 1–52 revealed two factors that each had an eigenvalue ≥1.0 and together explained more than 70% of the total variance of all the engagement metrics. The factor loadings (ie, the weights) of each engagement item on each factor are shown in table 3. The higher the loading of an engagement item, the more it is a pure measure of the factor. Comrey and Lee⁴² suggest that loadings >0.71 should be considered ‘excellent,’ loadings >0.63 considered ‘very good,’ and loadings >0.55 considered ‘good.’ In each engagement window, three engagement items loaded more strongly on one factor, and three items loaded more strongly on the other factor. Thus, one factor was constructed to comprise three self-focused tasks that participants were asked to perform each week: complete the assigned lesson, track their weight (at least once), and track their steps (at least once), and this will be referred to as ‘Lessons and Tracking Consistency.’ Similarly, the other factor was constructed to comprise the number of login sessions to the program website and the two group-focused activities (posting or replying to comments on the group board, and ‘liking’ others’ comments), and this will be referred to as ‘Logins and Group Participation.’ (Coach Conversations during weeks 1–16 did not strongly load on either factor and was not used during factor construction.)

Table 3.

Factor analysis of engagement metrics during the first year

Metric	Weeks 1–16		Weeks 17–52		Weeks 1–52
	Factor 1	Factor 2	Factor 1	Factor 2	Factor 1	Factor 2
Lessons completed	0.903	0.204	0.277	0.777	0.178	0.847
Weight tracked	0.814	0.232	0.098	0.854	0.189	0.831
Steps tracked	0.833	0.135	0.557	0.584	0.473	0.664
Coach conversations	0.444	0.434	–	–	–	–
Group conversations	0.263	0.875	0.847	0.290	0.854	0.303
Group posts liked	0.045	0.892	0.948	0.075	0.938	0.112
Log in sessions	0.335	0.807	0.714	0.523	0.718	0.514

Bold font indicates the highest loadings on each factor within each time window.

Regression analysis of engagement factors

Table 4 presents the results of the regression analyses exploring how the two engagement factors (Lessons and Tracking Consistency, Logins and Group Participation) predicted LMM-adjusted weight change from baseline at different time windows: engagement during weeks 1–16 predicting weight change at week 16 (table 4A) and engagement during weeks 1–16, weeks 17–52, and weeks 1–52 predicting weight change at 1 year (table 4B). In all analyses, the Lessons and Tracking Consistency factor did not predict weight loss, whereas the Logins and Group Participation did, particularly in the case of early engagement. A 1 SD increase along this factor during weeks 1–16 was associated 3.0 lb of additional weight loss at 1 year (β = −3.02, p=0.002), whereas the same increase along this factor during weeks 17–52 was associated with 2.5 lb of additional weight loss at 1 year (β = −2.53, p=0.01). (Neither engagement factor predicted weight change at 2 or 3 years.)

Table 4A.

Regression analysis of engagement factors predicting weight change* at 16 weeks

	β	SE	p Value
Engagement factors: Weeks 1–16
Lessons and Tracking Consistency	−0.71	1.26	0.57
Logins and Group Participation	−3.43	0.92	0.0002

*Weight change (in lb) calculated using linear mixed models, adjusting for participants’ baseline age and gender.

Table 4B.

Regression analysis of engagement factors predicting weight change* at 1 year

	β	SE	p Value
Engagement factors: Weeks 1–16
Lessons and Tracking Consistency	−0.29	1.85	0.97
Logins and Group Participation	−3.02	0.97	0.002
Engagement factors: Weeks 17–52
Lessons and Tracking Consistency	−0.92	1.35	0.50
Log ins and Group Participation	−2.53	1.15	0.03
Engagement factors: Weeks 1–52
Lessons and Tracking Consistency	−0.75	1.51	0.62
Log ins and Group Participation	−2.95	1.15	0.01

*Weight change (in lb) calculated using linear mixed models, adjusting for participants’ baseline age and gender.

Conclusions

The present paper serves both as a 3-year progress report of clinical outcomes of a digital translation of the DPP and as an exploration of the association between program engagement and weight loss. Significant reductions in weight and A1c were maintained at 1 year, 2 years, and 3 years relative to baseline. When two higher-order factors derived from factor analysis of engagement metrics were used to predict weight loss, one factor (comprising program website logins and group participation) was significantly associated with weight loss; the other factor (comprising the consistency of completing lessons, weighing in, and tracking steps) was not. Several implications emerge from this set of findings.

A first implication relates to the long-term effectiveness of digitally delivered intensive lifestyle interventions. Of the dozens of published translations of the DPP²¹ or the Finnish Diabetes Prevention Study⁴³ (see meta-analyses^11–13), only four—all delivered in person—have reported weight outcomes at 3 years. Sakane et al ¹⁴ reported an average weight loss of 4.0 lb, Jiang et al ¹⁶ reported an average weight loss of 2.4 lb, Gilis-Januszewska et al ¹⁷ reported an average weight loss of 2.5 lb, and Ramachandran et al ¹⁸ reported an average weight gain of 1.36 lb relative to baseline weight. Thus, the superior weight loss outcomes seen in the present digital DPP translation—originally structured to last just 1 year—are a testament to the effectiveness of this delivery format. The saliency and convenience of a digitally delivered intervention puts the tools of behavior change (ie, educational resources, tracking tools, coach advice, and peer support) literally ‘at one’s fingertips.’ This in turn increases the likelihood of long-term engagement and clinically meaningful outcomes. On the other hand, the gradual regression of body weight and A1c toward their preintervention values (cf table 1) illustrates another key finding from the intensive lifestyle intervention literature: that the likelihood of achieving lasting clinical benefit is greater when the primary intervention extends beyond the first year,^{44 45} or when a high-touch maintenance intervention is used.^{12 45}

A second implication relates to the statistical approach used to explore the relationship between online program engagement and clinical outcomes. Numerous prior studies have reported simple associations between isolated program features and weight loss, including lesson completion, frequency of weight and activity tracking, and number of conversations with the personal health coach or peer group.^22–27 Far fewer digital DPP translations have taken patterns of association among engagement metrics into account when predicting weight loss^28–32; their results, like ours, paint a more complex picture.

In a latent growth curve analysis, Jacobs et al ²⁸ reported a significant effect for the frequency at which meals were logged and a non-significant effect for the number of lessons completed on a 3-month weight loss (controlling for age, gender, and geographic location). In a backward stepwise regression analysis, Michaelides et al ²⁹ reported that the frequencies of weigh-ins and meals logged, but not the number of group posts, were significant predictors of a 6-month weight loss. In a multiple regression analysis, Chin et al ³⁰ reported that increased frequencies of logging weight, exercise, breakfast, and dinner were all positively associated with a 9-month weight loss (controlling for age, gender, and baseline BMI). In a multiple regression analysis, Sherifali et al ³¹ reported non-significant effects of website login frequency and lesson unit completion on a 1-year weight loss (controlling for age and gender). Finally, in a structural equation analysis, Kim et al ³² reported that increased group participation and food logging frequency were predicted in a 6-month weight loss.

Interestingly, neither the present study nor those studies just cited have identified online lesson module completion as a significant predictor of weight loss once other program behaviors (and key demographic variables) had been taken into account. This finding is noteworthy given that lesson completion is treated as a key proxy for overall program engagement by the CDC.^{21 34 39} In other words, although lesson completion is an easy metric to compute, it may not be the most important metric to consider when evaluating program effectiveness.

A third implication relates to the ‘order of operations’ around changing behavior in a digitally delivered intervention. That group-focused program behaviors and general website usage were more strongly associated with weight loss than self-paced lesson completion and tracking behaviors suggests that future digital programs might want to consider designing an experience that guides participants through a sequence of new habits: (1) consistently showing up (by logging in daily); (2) consistently opening up (by providing and receiving social support and accountability to and from their peer group); (3) consistently leveling up their knowledge (by completing lesson modules) and self-monitoring (by tracking weight, activity, and meals); and (4) consistently keeping it up (by being vigilant against relapse). In this approach, emphasis is placed primarily on participatory activities and secondarily on individual activities. The ‘power of peers’ as a key component of behavior-based weight loss interventions (both those delivered in person and digitally) has been the subject of systematic reviews and meta-analyses,^46–48 with some individual studies even finding support for the use of peer coaches as a viable alternative to professional coaches.^{49 50}

The present findings and conclusions should be viewed in light of study design and analytic limitations. First, the study sample was relatively small sample by the standard of more recently initiated digital DPP translations. Second, participant attrition increased over time, necessitating the use of LMMs during analysis. The percentage of enrolled participants who provided a weight at 3 years (102 out of 220; 46%) was, however, higher than that of Jiang et al ¹⁶ (834 out of 2553, 33%) and Gilis-Januszewska et al ¹⁷ (105 out of 262, 40%) at this same time point. Nevertheless, the present findings may represent an above-average picture of long-term clinical outcomes in a digital DPP, as individuals with below-average outcomes may have failed to voluntarily participate at later time points. Third, a control intervention was not used, which would have better contextualized weight change over time and the causal impact of the intervention in shaping it. A large-scale randomized controlled trial using the Omada Health Program is currently in the planning stages. Fourth, it should be understood that patterns of observed relationships among engagement metrics are driven by the availability and definition of those metrics. If additional engagement metrics had been available, the resultant factor analytic ‘portrait’ may have turned out differently. Additionally, specific program features (eg, curriculum structure) and program behaviors (eg, conversations with the health coach) may have not-so-subtle differences among digital DPP translations.

In summary, the present study adds to the digital DPP translation literature in two ways: by highlighting the potential for lasting clinical efficacy and by highlighting the need to clarify which aspects of program engagement are causally associated with weight loss. With more than 130 DPRP-registered organizations currently offering a partially or fully digital DPP experience, continued exploration of these two points is vital.