Abstract
Background
Primary care addresses obesity through physician oversight of intensive lifestyle interventions or referral to external programs with demonstrated efficacy. However, limited information exists on community program reach, effectiveness, and costs across different groups of participants.
Objective
To evaluate a scalable, community weight loss program using reach, effectiveness, and cost metrics.
Design
Longitudinal pre–post quasi-experiment without control.
Participants
Enrolled participants in Weigh and Win (WAW), a community-based weight loss program.
Intervention
A 12-month program with daily social cognitive theory-based email and/or text support, online access to health coaches, objective weight assessment through 83 community-based kiosks, and modest financial incentives to increase program reach.
Main Measures
Number of participants, representativeness, weight loss achievement (3%, 5% of initial weight lost), and cost of implementation.
Key Results
A total of 40,308 adults (79% women; 73% white; BMI = 32.3 ± 7.44, age = 43.9 ± 13.1 years) enrolled in WAW. Women were more likely than men to enroll in the program and continue engagement beyond an initial weigh-in (57% vs. 53%). Based on census data, African Americans were over-represented in the sample. Among participants who engaged in the program beyond an initial weigh-in (n = 19,029), 47% and 34% of participants lost 3% and 5% of their initial body weight, respectively. The average duration for those who achieved 5% weight loss was 1.7 ± 1.3 years. African American participants were more likely to achieve 5% weight loss and remain enrolled in the program longer compared to non-African American participants (2.0 ± 1.3 vs. 1.6 ± 1.2 years). Implementation costs were $2,822,698. Cost per clinically meaningful weight loss for African Americans ($257.97/3% loss; $335.96/5% loss) was lower than that for Hispanics ($318.62; $431.10) and Caucasians ($313.65; $441.87), due to the higher success rate of that subgroup of participants.
Conclusions
Weigh and Win is a scalable technology-supported and community-based weight loss program that reaches a large number of participants and may contribute to reducing health disparities.
KEY WORDS: behavioral weight Loss, public health, impact, incentives, cost
INTRODUCTION
Approximately 64% of primary care patients are overweight or obese,1 , 2 and two-thirds of those patients would like to discuss weight loss with their physician.3 Lifestyle weight loss approaches, overseen by physicians, are recommended for overweight patients with comorbidities and for obese patients,4 with the goal of facilitating clinically meaningful weight loss (i.e., 3–5% of initial body weight). This proportional reduction in body weight reduces triglycerides, blood glucose, and hemoglobin A1c levels, and is associated with reduced risk of cardiovascular disease, morbidity, and mortality.4 However, physicians often report insufficient time and resources as well as uncertainty related to facilitating these weight loss interventions. Therefore, guidelines recommend that healthcare systems support the referral of these patients to efficacious electronic or commercial weight loss programs delivered in community or clinical settings.4 – 8
The evidence underlying these referral guidelines is primarily from efficacy trials that emphasize internal rather than external validity. However, representativeness is rarely reported in efficacy trials and limits an interpretation of external validity.9 Similarly, and fundamental to clinical and community decision making, program implementation costs are rarely reported.9 The lack of information on the representativeness of participants and costs makes it difficult for physicians and healthcare systems to compare across weight loss intervention options.
In contrast to interventions tested in efficacy studies, publicly available interventions that promote physical activity, healthful eating, and behavioral strategies typically have broader inclusion criteria (e.g., no requirements for study run-in period or exclusion of participants with comorbidities), use program data for evaluation rather than intensive research assessments, and encourage potential participants to try out the program to determine whether it is a good fit for their lifestyle. These programs are appealing because they can reach a large population—and theoretically at a low cost. Unfortunately, little is known about community-based interventions that do not depend on interactions with healthcare providers to encourage weight loss, and few studies combine metrics of representativeness, effectiveness, and cost in a way that could be used to compare outcomes across programs and allow for informed decisions related to program uptake for health organizations or clinical referral for primary care physicians.10 , 11
The primary objective of this study was to report on the outcomes of a scalable, community-based weight loss program using reach, effectiveness, and cost metrics important for healthcare systems interested in referring patients to, and underwriting, interventions for community implementation. Outcomes were operationalized to include the proportion of participants who (a) lost weight (b) achieved a 3% weight loss, and (c) achieved a 5% weight loss. Program implementation costs were examined as cost per participant as well as cost per participant who achieved a 3% or 5% weight loss for those enrolled between 2011 and 2014. Finally, an exploratory aim was to determine differences in the representativeness of participants to the regional population and robustness of effects and costs per participant achieving a 3% or 5% weight loss within populations that typically experience disparities in obesity and weight-related chronic conditions based on racial and ethnic status.
METHODS
This study was a longitudinal pre–post, quasi-experimental study designed to evaluate the reach, effectiveness, and costs associated with Weigh and Win (WAW). WAW is a technology- and community-based weight loss program supported by Kaiser Permanente Colorado as a community benefit with the goal of providing a scalable, accessible evidence-based program that community residents could participate in for free. This study was part of an ongoing integrated research–practice partnership intended to improve weight loss interventions through externally validated research methods, while also providing practical information for program development and impact in worksite and community settings.12 – 20 The goal of integrated research–practice partnerships is to concurrently work towards moving the scientific field forward through ecologically valid research and improving practice through more rigorous evaluation. This partnership included scientific partners with expertise in translational obesity research and practice partners from incentaHEALTH™—the WAW delivery organization—and from Kaiser Permanente Colorado Community Benefit and Relations. The extant de-identified data and invoices for program delivery were provided to the research partners, who conducted all analyses independently of practice partners. The RE-AIM framework21 was used to guide assessment based on the individual-level indicators (reach, effectiveness, maintenance) and implementation costs. All study procedures were approved by the Virginia Tech institutional review board.
Intervention
The 12-month WAW program includes a website, objective weight assessment, daily social cognitive theory-based email and text message support, online access to health coaches, and modest financial incentives intended to increase program reach and retention.19 Incentives were provided based on a minimum of 5% of initial body weight loss ($15 per quarter) and up to 30% reduction ($150 per quarter). The intervention encouraged increased intake of fruit, vegetables, lean protein, and complex carbohydrates while monitoring portion sizes. Physical activity recommendations were graduated from walking to combined strength and cardiovascular training, with approximately 30 min of physical activity 5 to 6 days a week, beginning with a recommendation of 10 min of low-intensity activity 4 days per week. Community-based kiosks (n = 83) with a scale and a camera to document the weight loss process were set up in community centers, local retailers, libraries, and other settings easily accessible to community residents (defined as locations that were open to the public and included hours beyond 9:00 to 5:00). Participants were encouraged to weigh in at least once each quarter.
Participants
Participants from the initial delivery year (2011) through 2014 were included in the evaluation (Fig. 1). Participants were recruited through a combination of advertising at the kiosk site, opportunistic media within the communities (e.g., newspaper and television coverage), and online promotion. Of note, the top response from participants regarding where they had heard about the program was that they had learned about it when walking by a kiosk. Participants could enroll by visiting the program website, completing a registration which included self-reported age, gender, race, and ethnicity, and then completing an initial weigh-in at a program kiosk. Alternatively, participants could initiate enrollment at a program kiosk and then complete enrollment by filling in the registration at the program website. De-identified demographic, program engagement, and weight data were used for analyses. Participant records were excluded if the participant was <18 years old, or if they did not complete the enrollment process by weighing in at a kiosk. Because of the general benefits of regular physical activity and healthful eating, regardless of weight status, participation in WAW was not restricted to overweight and obese individuals; however, participants with a BMI < 25 were not eligible for weight loss incentives. Therefore, we report the overall program reach including all adult participants, but limit study outcome analyses to participants with a BMI ≥ 25.
Figure 1.
Participant flow diagram by quarter between 2011 and 2014.
Measures
Program reach was operationalized as the number of adult participants and representativeness of the sample compared to census gender, race, and ethnicity data from the counties that included a WAW kiosk. Enrollment duration was measured as the length of time between participants’ initial weigh-in and most recent weigh-in. Weight (kg) was measured objectively using a calibrated medical-grade 600-lb-capacity Tanita WB-110A scale (Class III NTEP certified) at each kiosk. Quarterly weight was defined as the weigh-in recorded closest to the quarter-end date relative to the initial weigh-in date, and was recorded as missing when no weigh-ins were completed in a given quarter. Percentage weight loss was calculated using initial body weight. Once a participant had achieved a clinically meaningful weight loss (i.e., 3% or 5%), maintenance was defined as maintaining that magnitude of weight loss for 6 or 12 months. However, participants were not eligible for incentives beyond the initial 12 months of participation, though they could continue to weigh-in, visit the website, and receive email and text message support. Weight data through 2015 were collected to allow for 12-month data on participants who enrolled at the end of 2014 and to capture additional maintenance data from those who enrolled in 2013 or earlier.
Costs
Costs were gathered from all program invoices that occurred between 2011 and 2014 related to program implementation. The invoiced costs were grouped into technical systems support, program delivery personnel, kiosk leasing, marketing personnel and activities, and program incentives.
Analyses
Program reach was examined using odds ratios (OR; 90% CI) comparing the WAW sample to relevant census data for the region. The sample remainder was used as the denominator for calculating the odds relative to each race category in the respective sample (e.g., the odds of being African American in the WAW sample vs. the respective odds of being African American in the census data). Univariate analysis of variance (ANOVA) was used to test WAW reach sample for baseline BMI differences according to gender, race, and ethnicity. Pairwise comparisons were used to identify the source of significant differences in the case of a race effect.
Participant information was reported for the total sample as well as stratified by those who completed only one weigh-in versus those who completed more than one weigh-in. Chi-square tests (χ2) and univariate ANOVA were used to test for race, ethnicity, gender, and age differences between adult WAW enrollees who weighed-in more than once vs. those who weighed-in only once. Adjusted residuals (res adj) were examined to identify the source(s) of significant χ2, and OR (90% CI) were calculated for race categories with the greatest absolute values of res adj using the remaining sample within the respective subgroup analysis as the reference group (e.g., OR for African Americans relative to all other participants collapsed as the reference group).
Effectiveness assessments among adult participants with a BMI ≥25 kg/m2 were conducted using a baseline-value-carried-forward procedure to account for missing data. Univariate ANOVA was used to compare baseline values for weight and BMI within sample subgroups stratified by gender, weight status (i.e., overweight vs. obese participants), race, and ethnicity for those who a) weighed-in only once, b) weighed-in more than once, c) lost 3% of their weight, and d) lost 5% of their weight. A series of chi-square tests and univariate ANOVAs were used to compare proportional outcomes within the sample overall, as well as stratified by gender, weight status, race, and ethnicity. In the case of a significant chi-square resulting from a test involving a categorical variable with more than two levels (e.g., race), OR (90% CI) were used to assess effect size for the group(s) with the greatest absolute values of res adj. The remaining sample within the respective subgroup analysis was used as the reference group (e.g., OR for losing 3% body weight among African Americans was calculated using all other race groups collapsed as the reference group). Significant F-statistics were decomposed using pairwise comparisons. The probability value of p < 0.05 was adjusted to p < 0.001 using the Bonferroni method as an indicator of statistical significance to control family-wise error resulting from multiple comparisons. Effect sizes or significant test statistics were reported as η2.
The cost per participant was calculated by dividing the total implementation costs by the total number of unique participants who were overweight or obese. While healthy-weight participants are able to participate in the community program, we did not attribute costs to participants with a BMI < 25. The cost per participant who lost 3% or 5% body weight was calculated by dividing the total implementation costs by the total number of participants in each group who met the criteria. To determine total costs per subgroup, the total number of participants within that subgroup was multiplied by the cost per participant who was overweight or obese. The costs of 3% and 5% weight loss per subgroup were calculated by dividing the total cost per subgroup by the total number of participants within the subgroup who met the criteria.
RESULTS
Reach
Figure 1 displays the flow of participants for inclusion in the present analysis. A total of 40,308 adults (79% women; 73% white) enrolled in WAW between 2011 and 2014. The mean (SD) participant age and BMI were 43.9 (13.1) years and 32.3 (7.4), respectively, with the majority (58%) of participants being obese. Table 1 presents descriptive statistics for the sample of WAW enrollees ≥18 years old. Participants completed a mean (SD) of 4.9 (12.2) weigh-ins over 0.44 (0.78) years and received $10.54 (34.48) in monetary incentives.
Table 1.
Mean (SD) Initial Age, Weight and BMI for All 2011–2014 Enrollees ≥18 Years Old
| N (%) | Age (years)* | Weight (kg) | BMI (kg/m2) | |
|---|---|---|---|---|
| Enrollees | 40,308 | 43.9 (13.1) | 90.6 (22.3) | 32.4 (7.2) |
| Gender† | ||||
| Men | 7620 | 44.3 (13.2) | 104.7 (23.4) | 32.7 (6.8) |
| Women | 31,753 | 43.9 (13.1) | 87.4 (20.7) | 32.3 (7.3) |
| Weight status | ||||
| Normal weight | 4527 (11) | 40.9 (13.5) | 63.4 (9.6) | 22.4 (1.7) |
| Overweight | 12,313 (31) | 44.2 (13.4) | 76.8 (9.6) | 27.4 (1.5) |
| Obese | 23,468 (58) | 44.4 (12.8) | 103.1 (19.8) | 36.9 (6.0) |
| Race‡ | ||||
| African American | 3079 (8) | 43.1 (12.5) | 96.7 (22.9) | 34.6 (7.8) |
| Asian | 585 (2) | 38.8 (11.6) | 76.1 (17.5) | 28.5 (5.9) |
| Caucasian | 29,278 (73) | 45.0 (3.2) | 90.6 (22.2) | 32.1 (7.1) |
| Native American | 663 (2) | 40.5 (12.2) | 93.4 (23.2) | 34.1 (7.5) |
| Other | 3542 (9) | 38.2 (11.4) | 88.6 (22.0) | 33.1 (7.3) |
| Ethnicity§ | ||||
| Hispanic | 7740 (19) | 39.6 (11.9) | 89.7 (22.0) | 33.4 (7.4) |
| Non-Hispanic | 29,260 (73) | 45.0 (13.2) | 91.0 (22.4) | 32.2 (7.1) |
*n = 938 (2%) did not report age
† n = 935 (2%) did not report gender
‡n = 3161 (8%) did not report race
§n = 3308 (8%) did not report ethnicity
The mean (SE) baseline BMI was higher among Hispanic participants than non-Hispanics [F (df) = 193.95 (1), p < 0.001; η2 = 0.005]. There were also significant differences in baseline BMI according to race [F (df) = 116.54 (5); p < 0.001; η2 = 0.015]. Pairwise comparisons indicated that African Americans had a greater mean (SE) BMI [34.6 (0.13) kg/m2] than all other races (p < 0.001), except for Native Americans [34.1 (0.28) kg/m2; p = 0.08]. Asian participants had a lower mean (SE) BMI [28.5 (0.30) kg/m2] than all others (p < 0.001) (Table 1).
The WAW sample had a significantly higher proportion of women [OR (95% CI) = 4.01 (3.91, 4.10); p < 0.001] and African Americans [OR (95% CI) = 4.78 (4.61, 4.96); p < 0.001] and a significantly lower proportion of Caucasians [OR (95% CI) = 0.42 (0.41, 0.42); p < 0.001] and Hispanic/Latinos [OR (95% CI) = 0.93 (0.90, 0.95); p < 0.001] than the census population (Table 2). A greater proportion of participants who weighed-in more than once were African American [res adj = 7.9; OR (90% CI) = 1.33 (1.24, 1.44); p < 0.001] than those who weighed-in only once, and a significantly greater proportion of those who weighed-in only once self-reported their race as “other” [res adj = −9.9; OR (90% CI) = 0.69 (0.65, 0.75); p < 0.001] than those who weighed-in more than once [χ2 (df) = 153.56 (5); p < 0.001; η2 = 0.062]. Further, a greater proportion of women weighed-in more than once than did men [χ2 (df) = 42.78 (1); p < 0.001; η2 = 0.033]. Differences in age and ethnicity distributions between those who weighed-in more than once versus those who weighed-in only once did not reach statistical significance.
Table 2.
Representativeness of Weigh-and-Win Participants Compared to Census Data (enrollee age ≥18 years; no BMI restriction)
| Total | Only 1 weigh-in | >1 Weigh-in | Census data | |
|---|---|---|---|---|
| N | 40,308 | 18,697 | 21,611 | 5,012,333 |
| % Female | 78.8 | 78.0 | 79.4 | 48.1 |
| Median age – women (years) | 53.5 | 53.5 | 52.0 | 41.8 |
| Median age – men (years) | 52.5 | 51 | 52.5 | 39.8 |
| % African American | 7.6 | 6.6 | 8.6 | 1.7 |
| % American Indian/Alaska Native | 1.6 | 1.8 | 1.5 | 1.4 |
| % Asian | 1.5 | 1.6 | 1.3 | 1.1 |
| % Caucasian | 72.6 | 72.6 | 72.6 | 86.3 |
| % Other race | 8.8 | 10.3 | 7.4 | 6.9 |
| % Hispanic or Latino | 19.2 | 20.0 | 18.6 | 20.4 |
Effectiveness
Table 3 presents overall descriptive statistics for the sample of participants with a BMI ≥ 25 kg/m2, as well as statistics stratified by gender and weight status. The mean (SD) weight loss was 2.1 (6.5) kg, and approximately 46% of the participants lost weight. Fifty-seven percent of participants weighed-in more than once, and 70% of these lost weight (Fig. 2). Twenty-seven percent and 19% of participants lost 3% and 5% of initial body weight, respectively. There were no gender differences in total weight lost or the proportion of participants losing 3% and 5% of body weight.
Table 3.
Impact Assessments by Gender and Weight Status
| Weight status | |||||||||
|---|---|---|---|---|---|---|---|---|---|
| Total Sample | Gender | Overweight | Obese | ||||||
| Male | Female | Total | Male | Female | Total | Male | Female | ||
| N Adults ≥25 kg/m 2 | 33,656 | 6678 | 26,246 | 10,949 | 2072 | 8595 | 22,707 | 4606 | 17,651 |
| N (%) 1 weigh-in only | 14,627 (44) | 3132 (47)† | 11,281 (43) | 4787 (44) | 994 (48) | 3715 (43) | 9840 (43) | 2138 (46) | 7566 (43) |
| M (SD) initial kg | 94.5 (20.7) | 106.9 (21.6)† | 91.1 (19.1) | 77.4 (9.6) | 88.5 (9.0) | 74.4 (7.3) | 102.9 (19.5)* | 115.5 (20.4)* | 99.3 (17.7)* |
| M (SD) initial BMI | 33.8 (6.5) | 33.6 (6.4) | 33.8 (6.6) | 27.5 (1.4) | 27.7 (1.4) | 27.5 (1.4) | 36.8 (5.9)* | 36.3 (5.9)* | 36.9 (5.9)* |
| N (%) ≥1 weigh-in | 19,029 ( 57 ) | 3546 ( 53 ) | 14,965 ( 57 ) † | 6162 ( 56 ) | 1078 ( 52 ) | 4880 ( 57 ) | 12,867 ( 57 ) | 2468 ( 54 ) | 10,085 ( 57 ) |
| M (SD) initial kg | 95.0 (21.3) | 108.3 (23.1)† | 91.8 (19.5) | 77.4 (9.3) | 88.6 (8.8) | 74.8 (7.4) | 103.4 (20.2)* | 116.9 (22.1)* | 100.1 (18.3)* |
| M (SD) initial BMI | 33.9 (6.7) | 33.8 (6.5) | 34.0 (6.8) | 27.6 (1.4) | 27.8 (1.4) | 27.6 (1.4) | 36.9 (6.1)* | 36.4 (6.1)* | 37.1 (6.2)* |
| N (%) lost weight | 13,413 (70) | 2568 (39) | 10,469 (40) | 4163 (68) | 743 (36) | 3277 (38) | 9250 (72)* | 1825 (40) | 7192 (41)* |
| M (SD) kg lost | 3.3 (7.7) | 3.7 (8.1) | 3.2 (7.6) | 1.8 (4.9) | 2.1 (4.8) | 1.7 (4.9) | 4.0 (8.7)* | 4.4 (9.1)* | 3.9 (8.6)* |
| N 3% weight loss | 9027 (27) | 1744 (26) | 7004 (27) | 2725 (25) | 486 (24) | 2134 (25) | 6302 (28)* | 1258 (27)* | 4870 (28)* |
| M (SD) initial kg | 95.9 (21.2) | 108.9 (22.9)† | 92.6 (19.5) | 77.7 (9.3) | 88.8 (8.2) | 75.1 (7.4) | 103.7 (20.1)* | 116.6 (22.0)* | 100.3 (18.1)* |
| M (SD) initial BMI | 34.1 (6.8) | 34.0 (6.5) | 34.2 (6.8) | 27.6 (1.4) | 27.8 (1.3) | 27.6 (1.4) | 37.0 (6.2)* | 36.4 (6.1)* | 37.1 (6.2)* |
| M (SD) kg lost | 6.9 (8.7) | 7.4 (9.0) | 6.8 (8.7) | 4.4 (5.2) | 4.9 (5.0) | 4.3 (5.3) | 8.0 (9.7)* | 8.3 (10.0)* | 7.8 (9.9)* |
| M (SD) years enrolled | 1.5 (1.3) | 1.3 (1.2) | 1.5 (1.3)† | 1.4 (1.3) | 1.4 (1.2) | 1.4 (1.2) | 1.6 (1.3)* | 1.3 (1.2) | 1.6 (1.3)* |
| M (SD) incentives ($) | 34.1 (55) | 32.3 (54.2) | 34.5 (55.2) | 27.4 (40.0) | 25.4 (39.8) | 27.8 (39.9) | 37.0 (60.0)* | 35.0 (58.6)* | 37.5 (60.5)* |
| N (%) maintained 6 mos. | 3054 (34) | 525 (30) | 2398 (34) | 866 (32) | 146 (30) | 674 (32) | 2188 (35) | 379 (30) | 1724 (35) |
| N (%) maintained 12 mos. | 1699 (19) | 279 (16) | 1330 (19) | 474 (17) | 85 (18) | 357 (17) | 1225 (19) | 194 (15) | 973 (20) |
| N (%) dropped after goal | 4380 (49) | 916 (53)† | 3344 (48) | 1336 (49) | 250 (51) | 1039 (49) | 3044 (48) | 666 (53)† | 2305 (47) |
| N (%) 5% weight loss | 6464 (19) | 1218 (18) | 5032 (19) | 1857 (17) | 312 (15) | 1472 (17) | 4607 (20)* | 906 (20)* | 3560 (20)* |
| M (SD) initial kg | 96.5 (21.4) | 109.9 (23.2)† | 93.3 (19.6) | 77.7 (9.1) | 88.7 (8.1) | 75.3 (7.4) | 104.1 (20.2)* | 117.2 (22.2)* | 100.7 (18.3)* |
| M (SD) initial BMI | 34.4 (6.9) | 34.3 (6.6) | 34.4 (6.9) | 27.7 (1.3) | 27.8 (1.3) | 27.7 (1.4) | 37.1 (6.3)* | 36.6 (6.2)* | 37.2 (6.4)* |
| M (SD) kg lost | 8.5 (9.5) | 9.1 (9.8) | 8.3 (9.4) | 5.5 (5.7) | 6.1 (5.5) | 5.4 (5.7) | 9.7 (10.4) | 10.1 (10.7) | 9.5 (10.3) |
| M (SD) years enrolled | 1.7 (1.3) | 1.5 (1.2) | 1.7 (1.3)† | 1.6 (1.3) | 1.6 (1.2) | 1.6 (1.3) | 1.7 (1.3) | 1.5 (1.2) | 1.8 (1.3)* |
| M (SD) incentives ($) | 50.0 (60.0) | 45.8 (59.9) | 47.2 (60.2) | 39.5 (43.0) | 39.2 (43.9) | 39.5 (42.8) | 50.0 (66.0)* | 48.1 (64.4) | 50.4 (65.8)* |
| N (%) maintained 6 mos. | 2343 (36) | 508 (42) | 2304 (46) | 623 (33.5) | 105 (34) | 489 (33) | 1720 (37) | 283 (31) | 1368 (38) |
| N (%) maintained 12 mos. | 1227 (19) | 274 (23) | 1297 (26) | 316 (17) | 59 (19) | 237 (16) | 911 (20) | 142 (16) | 724 (20) |
| N (%) dropped after goal | 2764 (423) | 543 (45) | 2046 (41) | 825 (44.4) | 135 (43) | 658 (45) | 1939 (42) | 425 (47) | 1463 (41) |
Values in bolded rows used as denominator for subsequently reported proportions within the column in the respective table section. Italicized numbers reflect the total sample in column and are used as denominator for subsequently reported italicized proportions within the column.
*Overweight vs. obese p ≤ 0.001 †Men vs. women p ≤ 0.001
Figure 2.
Histogram of participant weight loss (kg) for those who completed a second weigh-in.
Table 4 presents descriptive statistics for the sample stratified by race and ethnicity. African American participants had greater initial weight compared to non-African American participants [F (df) = 61.04 (5); p < 0.001; η2 = 0.009]. Further, African American participants had a significantly greater proportion of those losing 3% and 5% weight than did other groups [χ2 (df) ≥ 107.20 (5); p < 0.001; η2 ≥ 0.057]. African Americans were 37% more likely to lose 3% body weight [OR (90% CI) = 1.37 (1.26, 1.49); p < 0.001], 47% more likely to lose 5% body weight [OR (90% CI) = 1.47 (1.35, 1.61); p < 0.001], and about 25% less likely to drop out immediately following goal attainment [OR (90% CI) = 0.74 (0.64, 0.85) and 0.76 (0.65, 0.90) for those who lost 3% and 5%, respectively; p < 0.001] than participants who were not African American [χ2 (df) ≥ 13.67 (5); p < 0.001; η2 ≥ 0.044]. Also, the enrollment duration for African American participants who met the 3% and 5% weight loss goals was significantly longer than that of other participants meeting these weight loss goals [F (df) ≥ 12.35 (5); p < 0.001; η2 ≥ 0.010]. Hispanic participants had a higher initial weight than non-Hispanic participants [F (df) = 100.55 (1), p < 0.001; η2 = 0.003]. Hispanic participants who achieved 3% and 5% weight loss goals had a longer average enrollment duration than those who were not Hispanic [F (df) ≥ 38.66 (1); p < 0.001; η2 ≥ 0.005] (Table 4).
Table 4.
Impact Assessments by Race and Ethnic Group
| Race | Ethnicity | ||||||
|---|---|---|---|---|---|---|---|
| African American | Asian | Caucasian | Native American | Other | Hispanic | Non-Hispanic | |
| N Adults ≥25 kg/m 2 | 2796 | 386 | 24,267 | 579 | 3034 | 6785 | 24,178 |
| N (%) 1 weigh-in only | 1037 (37) | 181 (47) | 10,538 (43) | 269 (47) | 1556 (51) | 3037 (45) | 10,520 (44) |
| M (SD) initial kg | 99.8 (23.8)* | 82.7 (14.5) | 94.6 (20.4) | 94.5 (20.6) | 91.5 (20.7) | 92.0 (20.9) | 95.1 (20.5)‡ |
| M (SD) initial BMI | 35.5 (6.9) | 30.9 (4.7) | 33.5 (6.4) | 34.4 (6.6) | 34.2 (6.7) | 34.3 (6.8) | 33.6 (6.4) |
| N (%) ≥1 weigh-in | 1759 (63) | 205 (53) | 13,729 (57) | 310 (54) | 1478 (49) | 3748 (55) | 13,658 (56) |
| M (SD) initial kg | 98.4 (22.2)* | 83.0 (17.0) | 95.1 (21.0) | 97.9 (23.7) | 91.8 (21.3) | 92.9 (21.3) | 95.6 (21.2) |
| M (SD) initial BMI | 35.3 (7.7) | 31.2 (5.6) | 33.7 (6.6) | 35.7 (7.3) | 34.4 (6.8) | 34.6 (7.1) | 33.8 (6.6) |
| N (%) lost weight | 1182 (67.2) | 150 (73.2) | 9753 (71.0) | 221 (71.3) | 1001 (67.7) | 2555 (68.2) | 9692 (71.0) |
| M (SD) kg lost | 3.5 (8.8) | 2.9 (8.1) | 3.3 (7.6) | 3.5 (8.0) | 2.8 (7.6) | 3.2 (7.9) | 3.3 (7.7) |
| N (%) 3% weight loss | 909 (33)* | 98 (25) | 6489 (27) | 141 (24) | 627 (21) | 1786 (26) | 6444 (27) |
| M (SD) initial kg | 99.7 (22.8) | 83.2 (17.1) | 95.7 (20.8) | 99.2 (22.9) | 94.1 (22.6) | 94.2 (21.9) | 96.5 (21.1) |
| M (SD) initial BMI | 35.8 (8.3) | 31.3 (6.0) | 33.9 (6.5) | 35.8 (7.4) | 35.0 (7.1) | 34.9 (7.3) | 34.0 (6.6) |
| M (SD) kg lost | 7.2 (9.9) | 6.5 (7.9) | 6.9 (8.7) | 7.4 (9.6) | 6.9 (8.8) | 6.9 (8.8) | 6.9 (8.8) |
| M (SD) years enrolled | 1.9 (1.3)* | 1.4 (1.2) | 1.5 (1.2) | 1.5 (1.2) | 1.4 (1.2) | 1.7 (1.3)† | 1.5 (1.2) |
| M (SD) incentives ($) | 40.2 (60.7) | 29.1 (46.9) | 33.9 (54.6) | 33.0 (60.0) | 27.8 (51.0) | 35.1 (53.3) | 34.0 (56.0) |
| N (%) maintained 6 mos. | 397 (43.7) | 32 (32.7) | 2104 (32.4) | 54 (38.3) | 165 (26.3) | 375 (37.8) | 2087 (32.4) |
| N (%) maintained 12 mos. | 248 (27.3) | 13 (13.3) | 1153 (17.8) | 20 (14.2) | 82 (13.1) | 376 (21.1) | 1145 (17.8) |
| N (%) dropped after goal | 381 (41.9)* | 48 (49.0) | 3181 (49.0) | 71 (50.4) | 340 (54.2) | 832 (46.6) | 3167 (49.1) |
| N (%) 5% weight loss | 698 (25)* | 66 (17) | 4606 (19) | 109 (19) | 424 (14) | 1320 (20) | 4561 (19) |
| M (SD) initial kg | 98.6 (21.9) | 82.8 (16.0) | 94.5 (20.7) | 95.5 (22.2) | 91.0 (20.6) | 95.1 (22.3) | 97.1 (21.2) |
| M (SD) initial BMI | 35.8 (8.4) | 31.2 (4.5) | 34.2 (6.6) | 36.1 (7.1) | 35.3 (7.3) | 35.2 (7.6) | 34.3 (6.7) |
| M (SD) kg lost | 8.7 (10.4) | 8.6 (8.6) | 8.5 (9.4) | 8.8 (10.3) | 8.5 (9.7) | 8.3 (9.5) | 8.6 (9.6) |
| M (SD) years enrolled | 2.0 (1.3)* | 1.6 (1.2) | 1.6 (1.2) | 1.6 (1.1) | 1.6 (1.3) | 1.9 (1.3)† | 1.6 (1.2) |
| M (SD) incentives ($) | 51.5 (64.9) | 43.0 (51.8) | 47.0 (59.7) | 42.1 (64.9) | 40.3 (57.4) | 46.6 (57.4) | 47.3 (61.5) |
| N (%) maintained 6 mos. | 319 (46) | 25 (38) | 1610 (35) | 36 (33) | 127 (30) | 662 (50) | 1996 (44) |
| N (%) maintained 12 mos. | 195 (28) | 11 (17) | 822 (18) | 16 (15) | 55 (13) | 371 (28) | 1114 (24) |
| N (%) dropped after goal | 259 (37)* | 26 (39) | 1994 (43) | 44 (40) | 204 (48) | 519 (39) | 1911 (42) |
Values in bolded rows used as denominator for subsequently reported proportions within the column in the respective table section. Italicized numbers used as denominator for subsequently reported italicized proportions within the column
*African American vs. all others p ≤ 0.001
†Hispanic vs. non-Hispanic p ≤ 0.001
Costs
Total implementation costs over the study period were $2,822,698. The majority of costs were related to annual fees ($1,758,766), which included the maintenance and oversight of technical system support ($1,124,803), kiosk leasing ($349,500), and participant-related prizes and activities unrelated to weight loss ($248,151). Program implementation personnel costs were $383,119. Marketing personnel and activity costs were $344,054, and weight loss incentives distributed to participants with a BMI > 25 totaled $300,000. Finally, monthly fees that were primarily related to internet and short message service (SMS) use were $36,759.
Cost per enrolled participant who was overweight or obese (n = 33,656) was $83.87. The cost per participant achieving a 3% weight loss (n = 9027) was $312.70, compared to $436.68 for those who lost 5% of their initial body weight. Costs for men losing 3% and 5% body weight were $321.15 and $459.83, respectively, whereas costs for women were $314.28 and $437.45, respectively. Costs per African American participant achieving a clinically meaningful weight loss were $257.97 (3% loss) and $335.96 (5% loss). Costs for Hispanic ($318.62; $431.10) and Caucasian ($313.65; $441.87) participants were similar between those with 3% and 5% weight loss.
DISCUSSION
Approximately 19,000 WAW participants engaged in the program beyond enrollment tasks over the 4-year period. African American participants were more likely to enroll, extend their engagement beyond enrollment, lose a clinically meaningful amount of weight, and remain engaged longer than participants of other races. WAW also recruited four times the average number of participants who identified as Hispanic when compared to RCT weight loss trials22. These data are promising and may support the call for healthcare systems to provide interventions that attract populations that experience health disparities and can be scaled broadly.23 , 24
The average participant weight loss was lower than interventions tested in RCTs,25 but appears to compare favorably when considering the proportion of participants who achieved a clinically meaningful weight loss. The weight loss trial by Tate and colleagues reported that 22% of the participants who completed follow-up achieved a 5% weight loss with an internet-based educational program—which is similar to the WAW baseline-value-carried-forward proportion of 19%. However, when a doctoral-level therapist provided personalized behavioral therapy, the proportion of participants achieving 5% weight loss increased to 45%.26 A more intensive intervention, the Diabetes Prevention Program, showed that a lifestyle intervention resulted in 49% of the 1079 participants and 35% of the 204 African American participants achieving a 7% weight loss.27 Using only those who engaged beyond enrollment, 34% of the 19,029 WAW participants and 40% of the 1759 African American participants achieved a clinically meaningful weight loss—making comparisons much more favorable. Furthermore, the average percentage weight loss for those who exceeded the 5% criterion was 8.5% of initial body weight. Finally, while it is challenging to compare the cost per clinically meaningful weight loss due to the paucity of this information in the literature, the ∼ $84 cost per participant for the 12-month program appears to be low when compared to $258 for 6 months of Weight Watchers participation.8
This study is limited by a lack of a control group and ability to generalize beyond the program region. In addition, the assessment of maintenance was based on participants maintaining participation 6 to 12 months post achieving a 3% or 5% weight loss. To address this, we considered the 43% of the participants who lost 5% but did not have additional follow-up assessments as unsuccessful at maintenance. Finally, costs were included for 4 years of implementation, but some participants from 2014 received the portions of the program in 2015, indicating that the costs may be somewhat underestimated. However, we also did not attribute costs to participants with a BMI under 25, which may have contributed to an overestimation of costs.
CONCLUSIONS
The primary objective of this study was to examine reach, effectiveness, and cost metrics associated with WAW as a way to compare weight loss interventions and inform healthcare system decisions on the appropriateness of community-program referral. WAW (a) reached a large number of participants and may contribute to reducing health disparities experienced by African Americans based on our exploratory analyses, and (b) had a cost per participant that rates favorably against other publicly available weight loss programs,6 , 8 and (c) the cost per participant who achieved a clinically meaningful weight loss appears to be modest,28 though comparisons across weight loss interventions remain difficult due to a lack of information on costs across RCT-tested interventions.9 – 11
Acknowledgements
Funding for this project was provided by the Harold M. Maurer Distinguished Chair in Public Health Research Endowment at the University of Nebraska Medical Center and the Virginia Tech Fralin Center for Translational Obesity Research. Thanks to Dr. Wen You who provided feedback on early drafts of the manuscript to review conceptual clarity and appropriateness of statistical methods, and to Kaiser Permanente Colorado for allowing access to all de-identified participant data and invoices for program implementation costs.
Compliance with Ethical Standards
Conflict of Interest
Lia Schoepke is employed by IncentaHEALTH LLC. Todd McGuire is employed by, holds the patent to, and is a part owner of IncentaHEALTH LLC. All other authors declare that they have no conflict of interest.
References
- 1.Ahmed NU, Delgado M, Saxena A. Trends and disparities in the prevalence of physicians’ counseling on diet and nutrition among the U.S. adult population, 2000-2011. Prev Med. 2016;89:70–5. doi: 10.1016/j.ypmed.2016.05.014. [DOI] [PubMed] [Google Scholar]
- 2.Vallis M, Piccinini-Vallis H, Sharma AM, Freedhoff Y. Clinical review: modified 5 As: minimal intervention for obesity counseling in primary care. Can Fam Physician. 2013;59:27–31. [PMC free article] [PubMed] [Google Scholar]
- 3.Phillips SM, Glasgow RE, Bello G, et al. Frequency and prioritization of patient health risks from a structured health risk assessment. Ann Fam Med. 2014;12:505–13. doi: 10.1370/afm.1717. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Jensen MD, Ryan DH, Apovian CM, et al. 2013 AHA/ACC/TOS Guideline for the Management of Overweight and Obesity in Adults: A Report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines and The Obesity Society. Journal of the American College of Cardiology 2013. [DOI] [PubMed]
- 5.The Community Guide: Obesity prevention and control: Technology supported multicomponent coaching or counseling interventions to reduce weight and maintain weight loss. 2010. (Accessed July 10, 2016, at http://www.thecommunityguide.org/obesity/TechnologicalCoaching.html.)
- 6.Tsai AG, Wadden TA. Systematic review: an evaluation of major commercial weight loss programs in the United States. Ann Intern Med. 2005;142:56–66. doi: 10.7326/0003-4819-142-1-200501040-00012. [DOI] [PubMed] [Google Scholar]
- 7.Perrin EM, Vann JC, Lazorick S, et al. Bolstering confidence in obesity prevention and treatment counseling for resident and community pediatricians. Patient Educ Couns. 2008;73:179–85. doi: 10.1016/j.pec.2008.07.025. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Gudzune KA, Doshi RS, Mehta AK, et al. Efficacy of commercial weight-loss programs: an updated systematic review. Ann Intern Med. 2015;162:501–12. doi: 10.7326/M14-2238. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Harden SM, Gaglio B, Shoup JA, et al. Fidelity to and comparative results across behavioral interventions evaluated through the RE-AIM framework: a systematic review. Syst Rev. 2015;4:155. doi: 10.1186/s13643-015-0141-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Wieland LS, Falzon L, Sciamanna CN, et al. Interactive computer-based interventions for weight loss or weight maintenance in overweight or obese people. Cochrane Database Syst Rev 2012:CD007675. [DOI] [PMC free article] [PubMed]
- 11.Tate DF, Finkelstein EA, Khavjou O, Gustafson A. Cost effectiveness of internet interventions: review and recommendations. Ann Behav Med. 2009;38:40–5. doi: 10.1007/s12160-009-9131-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Estabrooks PA, Glasgow RE. Translating effective clinic-based physical activity interventions into practice. Am J Prev Med. 2006;31:S45–56. doi: 10.1016/j.amepre.2006.06.019. [DOI] [PubMed] [Google Scholar]
- 13.Zoellner J, You W, Almeida F, et al. The Influence of Health Literacy on Reach, Retention, and Success in a Worksite Weight Loss Program. Am J Health Promot 2015. [DOI] [PMC free article] [PubMed]
- 14.You W, Almeida FA, Zoellner JM, et al. Who Participates in Internet-Based Worksite Weight Loss Programs? BMC Public Health. 2011;11:709. doi: 10.1186/1471-2458-11-709. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Wilson KE, Harden SM, Almeida FA, et al. Brief Self-Efficacy Scales for Use in Weight-Loss Trials: Preliminary Evidence of Validity. Psychol Assess 2015. [DOI] [PMC free article] [PubMed]
- 16.Harden SM, You W, Almeida FA, et al. Does Successful Weight Loss in an Internet-Based Worksite Weight Loss Program Improve Employee Presenteeism and Absenteeism? Health Educ Behav. 2015;42:769–74. doi: 10.1177/1090198115578751. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Estabrooks PA, Bhagat SM, Almeida FA, Davy BM, You W. Understanding the impact of a weight loss program: The roles of reach, retention, and effectiveness. REFACS. 2014;2:6. doi: 10.18554/refacs.v2i2.1167. [DOI] [Google Scholar]
- 18.Davy BM, You W, Almeida F, et al. Impact of individual and worksite environmental factors on water and sugar-sweetened beverage consumption among overweight employees. Prev Chronic Dis. 2014;11:E71. doi: 10.5888/pcd11.130207. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Almeida FA, You W, Harden SM, et al. Effectiveness of a worksite-based weight loss randomized controlled trial: The worksite study. Obesity (Silver Spring) 2015;23:737–45. doi: 10.1002/oby.20899. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Almeida FA, Wall SS, You W, et al. The association between worksite physical environment and employee nutrition, and physical activity behavior and weight status. J Occup Environ Med. 2014;56:779–84. doi: 10.1097/JOM.0000000000000180. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health. 1999;89:1322–7. doi: 10.2105/AJPH.89.9.1322. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Pagoto SL, Schneider KL, Oleski JL, Luciani JM, Bodenlos JS, Whited MC. Male inclusion in randomized controlled trials of lifestyle weight loss interventions. Obesity (Silver Spring) 2012;20:1234–9. doi: 10.1038/oby.2011.140. [DOI] [PubMed] [Google Scholar]
- 23.Cossrow N, Falkner B. Race/ethnic issues in obesity and obesity-related comorbidities. J Clin Endocrinol Metab. 2004;89:2590–4. doi: 10.1210/jc.2004-0339. [DOI] [PubMed] [Google Scholar]
- 24.Marmot M. Health equity: the challenge. Australian and New Zealand journal of public health. 2012;36:513–4. doi: 10.1111/j.1753-6405.2012.00948.x. [DOI] [PubMed] [Google Scholar]
- 25.Franz MJ, VanWormer JJ, Crain AL, et al. Weight-loss outcomes: a systematic review and meta-analysis of weight-loss clinical trials with a minimum 1-year follow-up. J Am Diet Assoc. 2007;107:1755–67. doi: 10.1016/j.jada.2007.07.017. [DOI] [PubMed] [Google Scholar]
- 26.Tate DF, Wing RR, Winett RA. Using Internet technology to deliver a behavioral weight loss program. JAMA. 2001;285:1172–7. doi: 10.1001/jama.285.9.1172. [DOI] [PubMed] [Google Scholar]
- 27.Wing RR, Hamman RF, Bray GA, et al. Achieving weight and activity goals among diabetes prevention program lifestyle participants. Obes Res. 2004;12:1426–34. doi: 10.1038/oby.2004.179. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Diabetes Prevention Program Research G Within-trial cost-effectiveness of lifestyle intervention or metformin for the primary prevention of type 2 diabetes. Diabetes Care. 2003;26:2518–23. doi: 10.2337/diacare.26.9.2518. [DOI] [PMC free article] [PubMed] [Google Scholar]