Abstract
Background: The goal of this randomized controlled trial was to compare the relative effectiveness of a comprehensive lifestyle intervention delivered through the YMCA, JOIN for ME, with an enhanced version of this program that included on-site supervised group physical activity and opportunities for peer support, in decreasing adolescent weight status.
Methods: Sixty-six adolescents with BMI >85th percentile and absolute BMI <50 were randomly assigned to the JOIN for ME or the JOIN for ME enhanced program. Teens in both conditions attended 16 weekly, in-person group sessions, followed by four biweekly and four monthly maintenance sessions. The enhanced condition also included weekly, group-based physical activity sessions and challenges. Group sessions were led by YMCA coaches.
Results: Groups did not differ at baseline by age (14.7 + 1.6 years), sex (60.6% female), racial/ethnic minority status (37.7%), or weight (53.0% with severe obesity). Retention was 91% at 16 weeks and 82% at 10 months. Controlling for minority status and sex, there were no significant group differences over time for BMI (p = 0.15), BMI z-scores for age and sex (BMIz, p = 0.07), or percent overweight (p = 0.15). Across all participants, on average, BMI decreased by 1.4 kg/m2, BMIz decreased by 0.12, and percent overweight decreased by 8.8% at four months.
Conclusions: There were no significant differences observed in primary outcomes for adolescents randomized to the standard and enhanced versions of the JOIN for ME program. Although the absence of a control condition precludes attribution to the intervention, teens in both conditions demonstrated greater decreases in weight status than what was observed in an open trial of the original program.
Keywords: adolescent, community, intervention, obesity
Introduction
The prevalence of obesity in adolescents is alarmingly high and continues to increase, with 38.7% of 12- to 15-year-olds and 41.5% of 16- to 19-year-olds meeting criteria for overweight or obesity.1 Obesity during adolescence confers risk for a number of negative health and psychosocial outcomes2–5 and tracks strongly to adulthood,6 which in turn is related to increased rates of morbidity, including multiple types of cancer, and all-cause mortality.7,8 Despite the importance of addressing obesity during adolescence, it remains an understudied area compared with focus on adults and school-age children.9
The US Preventive Services Task Force (USPSTF) recently reissued a Grade B recommendation,10 initially made in 2010,11 for clinicians to screen children aged 6–18 years for obesity and refer identified youth to comprehensive multicomponent programs with >26 contact hours over a 2- to 12-month period that target changes in diet and physical activity and are supported by behavioral strategies.11 Reissue of this recommendation in 2017 highlights the continued importance of multicomponent family-based programs with broad accessibility for youth with obesity. However, the majority of children in the United States do not receive evidence-based treatment for obesity, with few such programs available to youth in general,12 and adolescents in particular.
Offering interventions in primary care and community settings are strategies for increasing accessibility of evidence-based weight control interventions for adolescents. To date, very few such programs exist. Among existing programs, an intervention delivered in primary care targeting young adolescents from primarily Latino backgrounds adopted a stepped down approach in which treatment contact was decreased based on weight loss success.13 Initially, adolescents, alone or with their parent, attended monthly face-to-face and bimonthly phone sessions focused on weight management concepts. Relative to enhanced usual care, comprising a usual visit and provision of psychoeducation materials on a monthly basis, the intervention led to significant reduction in BMI for boys (−0.7 BMI units), but not for girls.
A lifestyle intervention delivered by dietitians in a community health care setting in Australia to adolescents demonstrated decreases in standardized, but not absolute, BMI for completers at 12 and 24 months.14
Two additional recent efforts involved delivering treatment to adolescents through the school system. A randomized controlled trial that compared a behavioral intervention, including goal setting, motivational interviewing, and text support between meetings, delivered by a school health educator to a no treatment control condition showed no significant benefit related to participation in the intervention.15 Similarly, a study that examined the effectiveness of a 12-session school nurse-delivered cognitive behavioral therapy intervention combined with afterschool exercise showed no significant differences in BMI, percent body fat, or waist circumference in adolescents who received the intervention vs. those receiving the educational control condition.16 Thus, despite recent advances in delivery of community-based interventions, there are few effective models targeting adolescents.
JOIN for ME is a community-based pediatric weight control program developed as a novel collaboration between a national health care organization (UnitedHealth Group) and a community-based organization, Y-USA. Conceptual underpinnings of the program include social cognitive and family-based models, emphasizing the importance of behavioral strategies such as self-monitoring, goal setting, and positive reinforcement, combined with youth and caregiver involvement. The program also incorporates evidenced-based principles from the pediatric weight control literature, including specific dietary and physical activity prescription combined with supporting behavioral strategies.
To increase potential for large-scale dissemination, the program was designed to be delivered within a community setting by trained nonprofessional coaches. Findings from an open trial of the JOIN for ME program were promising for children aged 6–12 years (i.e., 4.3% decrease in percent overweight), with adolescents showing more modest reductions (1% decrease in percent overweight) in weight status.17,18
In an effort to improve the efficacy of JOIN for ME for adolescents, the research team made a number of modifications to the program. These modifications were based on a conceptual understanding of adolescent development combined with existing evidence from weight control interventions with teens.
Developmental theory suggests that identity formation and increased autonomy are important goals for the adolescent period,19 and peer influence plays a key role in both processes.20 Thus, the primary goals of the modifications to JOIN for ME were to increase peer support and adolescent autonomy related to weight management. The mechanism through which we attempted to enhance peer support was via group physical activity, which has been shown to be effective in previous work.21–23 Support for adolescent autonomy was augmented by having separate parent and teen group meetings,24 which differs from the standard implementation of the program in which parents and teens attend groups together. Based on these considerations, several modifications to the curriculum were made as detailed below.
We conducted a small randomized controlled trial to test the relative efficacy of this enhanced version of JOIN for ME compared with the original program. Given that our main interest was to determine whether the modified intervention produced better outcomes than the standard intervention, we did not include a no treatment control condition.
The primary study hypothesis was that adolescents randomized to the enhanced program would demonstrate greater reduction in weight status, as assessed by BMI z-scores for age and sex (BMIz), at the end of active treatment (4 months) and maintenance (10 months) compared with those randomized to the standard program. Secondary aims were to examine changes in physical activity, dietary intake, and group cohesion in adolescents randomized to the enhanced vs. standard programs. We hypothesized greater increases in physical activity in adolescents randomized to the enhanced condition given the integrated physical activity component and targeted efforts to foster group cohesion related to physical activity. We anticipated similar improvements in diet quality of teens in both groups given that they received the same dietary prescription.
Methods
Procedure
Adolescents were recruited via referrals from area pediatricians, direct mailings, and community postings in each of the communities immediately surrounding the three YMCA locations involved in this trial. In addition, study information was posted in targeted YMCA locations and included in already scheduled e-mail notifications to YMCA members. YMCA membership was not a requirement for study participation. A brief telephone screening was used to assess preliminary eligibility, which was confirmed during an in-person evaluation.
Adolescents were eligible if they were (1) 13–17 years; (2) had a BMI >85th percentile and an absolute BMI <50 kg/m2; (3) had at least one English-speaking parent available to participate in group sessions; and (4) agreed to study participation and random assignment. Teens were excluded if (1) they were involved in another weight loss program at the time of the screening; (2) had a medical condition that interfered with the prescribed dietary plan or participating in physical activity; (3) had a developmental delay that affected their ability to engage with intervention materials or within a group setting; or (4) were in treatment for a major psychiatric disorder, including an eating disorder, or diagnosed with a major psychiatric disorder at the time of screening.
Before randomization, adolescents and at least one parent attended an information session and completed a baseline assessment held at our research center. Parent consent and adolescent assent were obtained at the baseline assessment visit; subsequent study assessments occurred at the end of weekly sessions (4 months; end of active treatment) and at the end of the maintenance sessions, which included tapering to biweekly, then monthly sessions (10 months). At each assessment visit, adolescents were weighed and measured and completed questionnaires. Participants were compensated $50 for completing baseline and 4-month assessments and $75 for completing the 10-month assessment. Physical activity and dietary assessments were obtained for all participants at baseline and 4 months, but only for a subsample of participants at the 10-month assessment. The latter are not reported here.
As shown in Figure 1, 156 adolescents were phone screened, 79 of whom were eligible and moved forward to the in-person evaluation. An additional 13 teens did not move forward from the baseline assessment, leading to randomization of 66 adolescents into 11 treatment groups (n = 6 standard, n = 5 enhanced). Medical clearance for study participation was obtained from each teen's medical provider.
Figure 1.
Consort diagram.
Eligible adolescents were randomized to one of the two treatment conditions. Teens were individually randomized to treatment groups, consisting of five to seven teens per group. Families were notified of the group start date and informed of their randomization status at the initial group meeting. Randomization was conducted by the study statistician. To account for potential differences in treatment response based on weight status, participants were assigned to intervention using urn randomization25 within each YMCA branch, with weight status (low vs. high based on absolute BMI) used as a stratification factor.
On two occasions, there were too few participants to form two groups when the next round of intervention was scheduled to begin. On these occasions, a separate randomization was performed to assign all available participants to a single group. Despite this adjustment to the randomization scheme, the final sample was equivalent across groups on both number of participants and demographic characteristics. All study procedures were approved by the institutional review board at Rhode Island Hospital.
Common Intervention Components
Adolescents assigned to both treatment conditions received the same JOIN for ME curriculum, with the exception of how the parent sessions were delivered (see below). The common curriculum was modified slightly by program developers following the open trial to align number of contact hours with USPSTF guidelines. Before the start of the intervention, all caregivers were required to attend a group-based orientation session in which the basic program philosophy was discussed, expectations for family participation and attendance were reviewed, and a handbook with program content was provided.
Both the JOIN for ME and JOIN for ME enhanced interventions were 10 months in duration and included 16 weekly sessions (i.e., 4 months of active treatment), followed by four biweekly and four monthly sessions (i.e., maintenance); all sessions were ∼75 minutes in length, including time for individual weigh-in and check-in and a 60-minute group meeting. The total possible contact hours for the curriculum was 30 hours. The sessions were facilitated by a trained YMCA-based coach in a mixed-sex group format of five to seven adolescents and delivered at three branches of the YMCA of Greater Providence. To reduce the risk of contamination by coaches, each YMCA staff delivered one or the other intervention, but not both.
Parents of teens in both intervention arms were asked to attend a total of five sessions across active treatment and maintenance (sessions 1, 4, 8, 16, and 20). As noted above, parents were provided with comprehensive program materials during an initial informational meeting and were required to attend these particular sessions to provide continuity in contact, as well as to capture key program transitions [i.e., start of program (session 1), end of treatment and transition to biweekly maintenance (session 16), transition to monthly maintenance meetings (session 20)]. The content of parent sessions was identical in the two conditions. Family-based membership to the YMCA was provided for adolescents in both treatment conditions, but at different times during the intervention (described below).
Intervention content included self-monitoring, stimulus control, goal setting, reducing LESS foods (e.g., cookies, candy, sweetened beverages, fried foods, and chip) and increasing YES! foods (e.g., fruits, vegetables, lean protein), reducing screen time, increasing physical activity, improving sleep hygiene, and relapse prevention. Adolescents were given specific caloric target ranges depending on weight: 1400–1600 kcal/day for ≤113 kg, and 1600–1800 kcal/day for >113 kg. Calorie targets were incorporated as part of the original JOIN for ME curriculum and are consistent with other evidence-based behavioral interventions conducted with teens.26–28 Key behavioral targets included self-monitoring of dietary intake and physical activity, limiting LESS foods to two servings per day, limiting screen time to 2 hours/day, and weekly attendance at group sessions. Before the start of each session, adolescents were weighed by the coach to provide the basis for discussing weekly behavioral targets.
Unique Intervention Components
JOIN for ME
Parents were asked to check-in with their teens regarding weekly behavioral targets and provide reinforcement for progress toward these goals. In this treatment arm, parents attended the five group sessions together with their teens during which scheduled content, as well as progress with regard to program components, was addressed. Participants and their families received family memberships to the YMCA halfway through active treatment in week 8.
JOIN for ME enhanced
The JOIN for ME enhanced intervention followed the same weekly curriculum as JOIN for ME. In addition, there were three enhancements that were independent of the primary curriculum. First, adolescents randomized to JOIN for ME enhanced spent 60 minutes participating in physical activity with their treatment group either before or after the psychoeducation group session. The weekly active treatment structured group exercise sessions were facilitated by a YMCA-based coach (different from the group coach) and included ∼20 minutes of strength training and 30 minutes of cardiovascular activity, with 10 minutes devoted to warm-up and cool down. Adolescents in the group took turns selecting weekly activities and families received their YMCA family memberships the first week of the intervention.
Second, teens randomized to JOIN for ME enhanced set group-level physical activity goals with both individual and group-based incentives to increase group cohesion and teamwork. For the first half of active treatment (i.e., 8 weeks), adolescents earned points for (1) attending weekly group and physical activity sessions, (2) completing a diet and activity log, and (3) achieving modest weight loss (defined as at least half pound). Points were accumulated and traded for nominal gift cards to area stores (e.g., Target, Walmart).
In addition, adolescents worked together to identify a specific challenge to complete as a group (i.e., entering a 5K walk/run). Completing group-based challenges and achieving a group goal, such as collectively participating in a targeted amount of physical activity during the week, were reinforced with nominal group prizes, such as water bottles. Incentives were intended to both increase individual, as well as group motivation, by explicitly reinforcing targeted behaviors in the context of working toward success with the group-based challenge. Finally, consistent with supporting teen autonomy, parents of adolescents randomized to the enhanced treatment arm attended the five parent sessions in separate groups from their teens. These groups were facilitated by a YMCA coach who was different from the teen group leader.
Coach Training and Supervision
The procedure for training coaches to deliver the intervention followed the successful process used in the open trial of the JOIN for ME program.29 A senior member of the YMCA staff was responsible for identifying potential group coaches from existing personnel. Eligible staff were required to complete training to proficiency in the JOIN for ME intervention materials. This training was directed by YMCA leadership. Coaches subsequently attended a half day training during which members of the investigative team reviewed program philosophy, principles of behavioral weight control, adolescent development, details of the research protocol, including standard communication with study participants, and strategies for managing group communication.
A second 2-hour meeting was held after the start of group sessions to review treatment lessons and address specific challenges. Weekly group supervision calls were held throughout the duration of the study and were conducted separately for coaches conducting the standard and enhanced treatment conditions. Coaches overseeing weekly group exercise sessions were asked to participate in supervision calls for the enhanced condition.
To assess fidelity of intervention delivery, weekly group sessions were videotaped for both treatment conditions. Of the total 176 recorded sessions, 20% (n = 36) were randomly selected for fidelity coding by a primary coder to assess (1) adherence to the weekly JOIN for ME curriculum content, rated yes, partial, or no; (2) general coach characteristics (i.e., interpersonal skills and ability to effectively manage group interactions), rated on a scale from 1 (not at all) to 5 (a lot); and (3) overall coach expertise, rated on a scale from 1 (novice) to 9 (advanced). Scores of 1–3 were categorized as novice, 4–6 as intermediate, and 7–9 as advanced. Fifty percent of the identified videotapes were coded for content adherence by a secondary coder (n = 18). Inter-rater reliability was assessed using a kappa statistic.
Measures
Demographics were measured at baseline only, and weight-related outcomes were measured at baseline, 4-month (end of active treatment), and 10-month (maintenance) time points. As noted above, measures of physical activity and diet were completed by all participants at baseline and 4 months only. Group cohesion was measured at 4 months and 10 months to capture evolution of relationships during intervention delivery. To facilitate completion, the 4- and 10-month assessments were conducted at the YMCA immediately preceding or following the scheduled group session. Assessments of physical activity and diet were completed during the week following the end of active treatment to be independent of intervention activities. A majority of follow-up assessments were conducted by research staff who were blinded to treatment condition; the exceptions being a small number of participant scheduling challenges that resulted in assessments by unmasked staff.
Demographics
Adolescent age, sex, qualification for free/reduced price meals, and race/ethnicity, as well as parent age, sex, education, and race/ethnicity were collected by parent report.
Anthropometrics
Weight was measured using a digital scale in street clothes, without shoes, and height was measured using a wall-mounted stadiometer at baseline and free-standing stadiometer to accommodate off-site (i.e., YMCA-based) evaluations at 4- and 10-month assessments. Digital scales were calibrated on a routine basis. Weight and height were used to calculate BMI (kg/m2) for parents and adolescents. Additionally, BMIz30 and percent overweight were calculated for teens. Percent overweight was calculated as [(Teen's BMI − BMI at 50th percentile for age and sex)/50th percentile BMI for age and sex) × 100]. Waist circumference was measured only in adolescents to the nearest 0.1 cm at the level of the iliac crest and anteriorly, at the umbilicus.
Dietary intake
The Block Kids Food Screener (BKFS; NutritionQuest, Berkley, CA) was used to measure usual food group intake based on the previous week's recalled intake. This screener captured reported increase in consumption of fruits, vegetables, and whole grains, and decrease in saturated fat, added sugar, and sugar-sweetened beverage, consistent with the intervention focus on increasing YES! foods and decreasing LESS foods. The BKFS has been validated for use with middle and high school students.31
Physical activity
The SenseWear Mini (MINI) armband objectively captured physical activity over 7 days. This physical activity monitor provides estimates of daily energy expenditure, and exercise duration and intensity using a triaxial accelerometer and a unique combination of heat sensors (i.e., heat flux, galvanic skin response, skin temperature, and near body temperature). Previous studies using both indirect calorimetry32 and doubly labeled water33 support the validity of the MINI to measure physical activity in pediatric populations. Adolescents were asked to wear the device for 7 days.34 Variables of interest included average daily minutes spent in moderate-to-vigorous physical activity (MVPA) and percent time sedentary.
Group cohesion
The Youth Sports Environment Questionnaire35 was used to assess participant perception of cohesion in their treatment group. This 18-item questionnaire includes two subscale scores, task and social cohesion. Task cohesion relates to the extent to which individuals feel that the group works well together and is committed to the same goals, whereas social cohesion assesses contact outside scheduled group times. Items are rated on a 10-point Likert scale. Scores for each subscale (task and social cohesion) ranged from 8 to 80.
Statistical Analysis
Analyses were conducted in SAS 9.4 (SAS Institute, Inc., Cary, NC). Multilevel models with random intercepts were used to model change in outcomes across baseline, post-treatment (4 months), and follow-up (10 months). Given evidence of differences across sex and minority status in weight and weight loss outcomes,36,37 fixed effects included time and treatment group; sex and minority status (White/non-White) were included as a priori-identified covariates in all models. In models assessing dietary outcomes, total caloric intake was included as an additional covariate. Group differences over time were tested by including a group × time interaction term in each model. Where interaction terms were statistically significant, models were stratified by group and the effect of time was examined.38 Probing of interactions for count outcomes was based on the methods put forth by Dawson.39
Weight-related outcomes (BMI, BMIz, percent overweight, and waist circumference) were modeled across baseline, end of treatment (4 months), and follow-up (10 months). Physical activity (total time spent engaging in MVPA), percentage of time spent engaging in sedentary behavior, and dietary outcomes [servings of fruit (cups), vegetables (cups), whole grains (ounces), saturated fat (grams), added sugar (teaspoons), and sugar-sweetened beverages (frequency)] were examined from baseline to the end of treatment.
Participants were included in the physical activity analyses if they had between 3 and 7 days of physical activity data, with at least eight valid hours wear time per day. Time spent in MVPA (minutes) was positively skewed and log-transformed. Fruit, vegetables, whole grains, saturated fat, and added sugar all were positively skewed and square-root transformed. A gamma distribution was used to model frequency of sugar-sweetened beverages.
This treatment development study was powered to detect changes in primary weight outcomes. With a sample size of 66, we had 91% power to detect a 0.1 [standard deviation (SD) = 0.15] difference in change in BMIz.
Results
Participants
As shown in Table 1, a total of 66 adolescents with overweight/obesity, aged 13–17 years [mean (M) = 14.7 ± 1.6], were randomized to participate in the JOIN for ME (n = 33) or the JOIN for ME enhanced (n = 33) intervention. There were no statistically significant group differences in baseline characteristics. A total of 60 participants (91%) completed the 4-month assessment (n = 30 in each group), whereas 54 participants (82%) completed the 10-month follow-up (n = 26 in JOIN for ME; n = 28 in JOIN for ME enhanced).
Table 1.
Baseline Characteristics of Teens Randomized to the JOIN for ME vs. the JOIN for ME Enhanced Program
| Total sample (n = 66) | JOIN for ME (n = 33) | JOIN for ME enhanced (n = 33) | |
|---|---|---|---|
| Age, mean ± SD, years | 14.7 ± 1.6 | 14.7 ± 1.7 | 14.8 ± 1.4 |
| % Female | 60.6 | 58.6 | 63.6 |
| Race/ethnicity, % | |||
| Non-Hispanic, White | 62.3 | 64.5 | 60.0 |
| Non-Hispanic, Black | 11.5 | 6.5 | 16.7 |
| Non-Hispanic, Other | 8.2 | 9.7 | 6.6 |
| Hispanic, all races | 18.0 | 19.3 | 16.7 |
| Maternal education, % | |||
| High school graduate or less | 14.1 | 25.8 | 3.0 |
| Some college/vocational degree | 29.7 | 32.3 | 27.3 |
| College degree | 26.6 | 16.1 | 36.4 |
| Graduate degree | 29.7 | 25.8 | 33.3 |
| % Free or reduced price meal qualification | 31.7 | 31.0 | 32.3 |
| Parent marital status (% married) | 61.9 | 61.3 | 62.5 |
| BMI, mean ± SD, kg/m2 | 34.6 ± 6.6 | 33.8 ± 6.5 | 35.4 ± 6.7 |
| BMIz, mean ± SD | 2.2 ± 0.5 | 2.2 ± 0.6 | 2.3 ± 0.5 |
| Percent overweight, mean ± SD | 77.3 ± 32.7 | 75.0 ± 33.0 | 79.6 ± 32.8 |
| Baseline weight status, % | |||
| Overweight (85 ≤ BMI <95th percentile for age/sex) | 10.6 | 9.1 | 12.1 |
| Obesity (BMI ≥95th percentile for age/sex) | 36.4 | 42.4 | 30.3 |
| Severe obesity (≥120% of 95th percentile for age/sex) | 53.0 | 48.5 | 57.6 |
| Parent BMI, mean ± SD, kg/m2 | 32.9 ± 6.4 | 32.9 ± 5.9 | 33.0 ± 6.9 |
BMIz, BMI z-scores for age and sex; SD, standard deviation.
Examination of completion rates across the 11 different treatment groups at each time point revealed no clear pattern, with 100% completion for the majority of groups at 4 months (range = 75%–100%) and rates ranging from 60% (one standard group) to 100% (two standard and one enhanced) at 10 months. Across the intervention, participants in both conditions attended a similar number of the weekly active treatment sessions (JOIN for ME: 12.3 ± 3.4; JOIN for ME enhanced: 11.2 ± 3.4; p = 0.18). There were no differences in attendance for the four biweekly or four monthly maintenance sessions across conditions (biweekly: JOIN for ME: 1.9 ± 1.3; JOIN for ME enhanced: 1.8 ± 1.3; p = 0.71; monthly: JOIN for ME: 1.7 ± 1.5; JOIN for ME enhanced: 1.4 ± 1.5; p = 0.46).
Coach Treatment Fidelity
Inter-rater reliability ratings for content delivered were acceptable for both the standard and enhanced conditions (κ = 0.72 and κ = 0.87, respectively). Coaches delivering the standard intervention adhered to 96% (±0.9) of content, whereas enhanced treatment coaches adhered to 93% (±1.8) of content. Coaches delivering the standard intervention had significantly higher ratings for two characteristics: smooth transitions between activities (p < 0.05) and ability to deliver content without reliance on written materials (p < 0.05). Standard treatment coaches also were rated as more expert than enhanced treatment coaches (MS = 7.4, SDS = 0.09 vs. ME = 6.3, SDE = 1.2; p < 0.001). There were no significant differences between coaches delivering standard and enhanced treatments on any of the remaining seven dimensions.
Weight-Related Outcomes
Similar patterns were observed for all outcomes (Fig. 2A–D). After controlling for minority status and sex, there were no significant group differences over time for BMI (p = 0.15), BMIz (p = 0.07), or percent overweight (p = 0.15), indicating no overall group differences in weight outcomes. Across all participants, on average, BMI decreased by 1.4 kg/m2, percent overweight decreased by 8.8%, and BMIz decreased by 0.12 at four months.
Figure 2.
Change in (A) BMI, (B) BMIz, (C) percent overweight, and (D) waist circumference in JOIN for ME vs. JOIN for ME enhanced. BMIz, BMI z-scores for age and sex. (Includes participants who provided data at all three time points.)
After controlling for minority status and sex, there was evidence of group differences in change in waist circumference (p = 0.03). After stratifying by group, teens randomized to JOIN for ME experienced a significant reduction in waist circumference at 4 months (b = −2.91, p = 0.01), whereas those randomized to JOIN for ME enhanced did not (b = 0.007, p = 0.99). These improvements were not maintained, as the average waist circumference of the JOIN for ME participants at 10-month follow-up was not significantly different from baseline (b = −1.8, p = 0.13), and participants randomized to JOIN for ME enhanced demonstrated a significant increase in waist circumference compared with baseline (b = 2.1, p = 0.03).
Physical Activity Outcomes
Controlling for sex and minority status, treatment condition did not significantly predict total time spent engaging in MVPA or percentage of time spent engaging in sedentary behavior (ps > 0.26; Table 2).
Table 2.
Estimates of Fixed Effects Predicting Physical Activity from Baseline to the End of Treatment (n = 49)
| Estimate | p | 95% Confidence interval |
||
|---|---|---|---|---|
| Lower bound | Upper bound | |||
| Total time: MVPA | ||||
| Intercept | 2.53 | <0.001 | 2.42 | 2.64 |
| Treatment | 0.03 | 0.75 | −0.12 | 0.19 |
| Time | −0.07 | 0.26 | −0.20 | 0.06 |
| Time × treatment group | 0.11 | 0.29 | −0.10 | 0.32 |
| Percent total time: sedentary behavior | ||||
| Intercept | 0.46 | <0.001 | 0.20 | 0.73 |
| Treatment | 0.01 | 0.90 | −0.28 | 0.30 |
| Time | −0.003 | 0.98 | −0.24 | 0.23 |
| Time × treatment group | 0.02 | 0.90 | −0.35 | 0.40 |
MVPA, moderate-to-vigorous physical activity.
Dietary Outcomes
Controlling for sex and minority status, neither treatment group nor the interaction between group and time emerged as significant predictors of reported fruit, vegetable, whole grain, saturated fat, added sugar, or sugar-sweetened beverage intake (data not shown). Adolescents in JOIN for ME and JOIN for ME enhanced reported consuming approximately two cups of fruits/vegetables per day at baseline (2.0 ± 1.1 vs. 1.8 ± 1.2), which did not change significantly by the end of active treatment. Reported intake of added sugar was fairly low for participants in both conditions (6.2 + 3.6 vs. 6.2 + 4.9 teaspoons) and did not decrease significantly during active treatment. The same was true for consumption of sugar-sweetened beverages, with adolescents in JOIN for ME (0.12 ± 0.16) and JOIN for ME enhanced (0.17 ± 0.21) reporting low consumption at baseline, which remained stable at the end of active treatment.
Group Cohesion
There was no difference between groups in task cohesion at 4 months (52.3 ± 15.1 vs. 45.6 ± 13.9; p = 0.08). There was a significant difference between groups at 10 months (50.2 + 15.8 vs. 40.9 + 16.4; p = 0.03), suggesting that participants in JOIN for ME experienced greater group cohesion around group goals and group functioning compared with those randomized to JOIN for ME enhanced. Social cohesion scores were comparatively lower in both groups, and group differences were not statistically significant at 4 months (22.3 + 15.5 vs. 19.5 + 12.0; p = 0.45) or 10 months (26.0 + 18.0 vs. 19.1 + 12.8; p = 0.11).
Discussion
In summary, the standard and enhanced versions of the JOIN for ME intervention did not lead to significantly different reductions in weight status at the end of the 4-month long active treatment phase. Waist circumference showed a significant decrease for teens randomized to the standard program at 4 months, but not among those in the enhanced condition; however, these improvements were not maintained at 10 months. Although the program included clear prescriptions related to improving diet quality and increasing physical activity, no significant changes were observed in dietary or physical activity behaviors in either condition. Furthermore, contrary to expectations, task-related group cohesion was significantly higher in the standard vs. the enhanced group at 10-month follow-up.
Contrary to hypotheses, an enhanced version of the JOIN for ME program did not lead to greater decreases in weight status for adolescents compared with the standard program. Reductions in weight status appear greater for treatment conditions in the current study (8.8%) compared with the 1% decrease in percent overweight observed in 13- to 17-year-olds who participated in the open trial.18 One potential explanation is the updated JOIN for ME curriculum, which includes more contact hours than the originally tested program and aligns with USPSTF guidelines. While not as robust as improvements documented in efficacy trials conducted in more controlled environments,26,27 the decreases observed in adolescents who received either JOIN for ME intervention are comparable to other community-based interventions for adolescents, including the Loozit trial, which reported stabilization of absolute BMI and a modest decrease of 0.09 BMIz units at 12 months.40
Community-based programs tend to be less intensive with far fewer in-person contact hours than controlled efficacy trials, which may account for these differences. Reduction of absolute BMI from 1.0 to 1.5 U has been referenced as a meaningful decrease in pediatric populations, which was achieved for both the JOIN for ME programs at 4 months.13 It is important to note that given the absence of a no treatment control condition, it is possible that the observed reductions in weight status are attributable to an unmeasured secular trend, such as unidentified changes in school policy promoting positive changes in diet and activity.
The only significant group effect observed was in the measure of waist circumference, with adolescents randomized to the standard JOIN for ME program demonstrating decrease at the end of the active program, which was not retained at 10 months. In contrast, adolescents randomized to the enhanced condition showed no decrease at the end of treatment and actually demonstrated significant gains in waist circumference by the 10-month follow-up. This outcome is counter to expectations, and the reasons for this finding are unclear, although it is aligned with the lack of sustained changes in weight status at 10 months. Given the number of statistical tests conducted, it is possible that the group difference observed in waist circumference is a spurious finding.
It was expected that participation in group-based physical activity along with group-based incentives and challenges would contribute to greater program engagement, greater group cohesion, an overall increase in MVPA, and an associated improvement in weight loss outcomes. Unexpectedly, adolescents in the enhanced condition did not report higher levels of either task or social cohesion at the end of treatment. To the contrary, at the 10-month assessment, adolescents in the standard condition reported significantly higher levels of task cohesion.
It appears that our efforts to enhance the sense of a cohesive group through additional activities were not successful. It is possible that adolescents experienced these efforts as contrived, which may have hampered more organic development of social connections. The enhanced condition was also more burdensome due to the increased time commitment (i.e., additional 60 minutes weekly), which may have negatively impacted adolescent engagement with prescribed activities. Alternatively, this may be a reflection of differences in efficacy of facilitators (based on fidelity ratings).
Treatment engagement as assessed via attendance at weekly sessions was comparable across treatment conditions during both the intensive and maintenance phases of treatment. Participants attended ∼70%–75% of sessions during the weekly intervention, which compares favorably with the 70% recommended benchmark for an adolescent weight control trial.41 However, attendance at biweekly and monthly maintenance sessions dropped sharply for participants in both treatment arms, to between 45% (standard) and 48% (enhanced) at biweekly and 43% (standard) and 35% (enhanced) monthly. Decreased attendance over the course of the maintenance phase of the program may account for the lack of sustained reductions in weight status over time.
Identifying strategies to support retention through the duration of evidence-based interventions poses a significant challenge. A recently completed adolescent weight loss program identified early response to treatment as a strong predictor of retention (SHINE), highlighting the importance of maximizing initial changes.42
Other potential approaches are informed by recently completed qualitative work in which the researchers conducted in-depth individual interviews with a small sample of adolescents (n = 7) who completed a teen program of similar length to JOIN for ME. When asked about strategies to enhance retention, these teens uniformly endorsed the option for coaches to maintain contact between sessions via text messages, noting that this would help to sustain an individual connection, as well as serve as a reminder of program goals. Text messaging has been identified as acceptable to teens, and the strategy has been related to decreased dropout from pediatric weight control intervention.43–45
There were no significant differences observed in physical activity or reported dietary intake. In terms of physical activity, expected increases in overall physical activity participation were not observed for the enhanced condition. As noted above, it is possible that adolescents randomized to the enhanced treatment relied on the provided structure for physical activity and did not increase their physical activity outside the treatment at the level anticipated (either during or after). It may also reflect differences in efficacy of facilitators.
In contrast to physical activity outcomes, between-group differences were not anticipated for dietary intake, as adolescents randomized to both treatment conditions received the same dietary prescription. Failure to observe improvements in dietary intake (i.e., increased fruit and vegetable intake, decreased sweetened beverage consumption) over time and across conditions may reflect a lack of change in these domains, floor effects for consumption of sugar-sweetened beverages and added sugars, or measurement limitations of the dietary intake screener.46 Compared with recommended national dietary guidelines,47 reported fruit and vegetable consumption was below the recommendation, whereas reported added sugar and sugar-sweetened beverage intake was consistent with the recommendation.
In summary, it is possible that our measures of dietary intake and physical activity were insensitive to changes in behaviors over time or that the timing of administration failed to capture such changes. Future research would benefit from incorporating interviews with adolescents during and at completion of intervention delivery to gain insight into changes contributing to decreases in weight status.
Findings must be considered in light of a number of limitations. Given that this was a treatment development study, the sample size was small, limiting our ability to examine differences in treatment response by baseline variables, such as level of obesity. Furthermore, because the aim of the study was to test the relative efficacy of the enhanced program compared with the standard program, there was no true control condition, which limits our ability to draw conclusions regarding program effectiveness. In addition, measures of diet and physical activity were only obtained at the end of treatment, precluding our ability to examine changes in these secondary outcomes over time.
Furthermore, two coaches were exclusively responsible for delivering the standard intervention and two for delivering the enhanced, leading to potential confounding of outcomes by coach. High treatment fidelity was observed across treatment arms, suggesting that intervention content was delivered as intended; however, coaches in the standard condition received significantly higher ratings on a subset of domains, which may have impacted outcomes. The small sample size and number of nesting units precluded our ability to add a clustering term to account for assignment of coaches in our analyses, and future work would benefit from specific investigation of what facilitator characteristics and/or skills contribute to optimal weight loss, physical activity, and dietary change outcomes.48
An additional consideration is the challenges experienced with the randomization scheme, which could have impacted allocation of teens to the two treatment conditions. Consistent with the study aim to test a disseminable community-based weight loss program for adolescents, treatment groups in both conditions were mixed sex. There are some data to suggest that female adolescents benefit more from same-sex mental health and lifestyle intervention groups, whereas male adolescents demonstrate better outcomes when assigned to mixed-sex groups.49–52 Future research would benefit from a more focused test of whether weight loss, physical activity, and dietary changes are greater if JOIN for ME is offered in a same-sex group format.
A final consideration is the decline in engagement observed during the maintenance phase. This is commonly observed in weight control interventions53 and has significant implications for maintenance of intervention effects. Future research should focus on identifying and testing strategies to sustain participation during the maintenance phase, including through use of text messaging or other digital media.
Despite these limitations, findings from this small randomized controlled trial demonstrate that a community-based weight control intervention led by YMCA coaches was delivered with fidelity and led to greater decreases in weight status in an adolescent sample compared with the open trial of the original program.17,18 Notably, there was no benefit associated with an enhanced condition that included greater attention to physical activity and group cohesion. There are a limited number of scalable weight management treatment options available for adolescents, despite the number of teens needing services. An intervention such as the standard JOIN for ME program that can be widely offered in community organizations and reliably delivered by lay coaches has significant implications.
Even so, there are several important considerations that need to be addressed in advance of such efforts. Replicating findings—both treatment fidelity and weight outcomes—with a larger sample size is critical. Developing effective strategies to enhance engagement through a longer intervention period, thereby potentially increasing treatment impact, also is critical. Assessing the sustainability of the intervention within the YMCA setting is important. Related to sustainability, gaining a better understanding of key implementation metrics, including the feasibility of delivering the program given space limitations, typical staffing patterns, and other programming at the YMCA, as well as acceptability of the program for eligible families and the organization will be essential for successful and sustained implementation.54 Finally, psychometrically valid assessment of these metrics would provide critical information necessary for larger scale dissemination of the JOIN for ME program for adolescents.
Acknowledgments
The authors are grateful to the families who participated in the study and the YMCA coaches who delivered the intervention. ClinicalTrials.gov no. NCT02426346.
Funding Information
This study was supported by Grant no. R34DK10098 from the National Institutes of Health (NIH) to E.J.
Author Disclosure Statement
E.J. is a consultant for Weight Watchers International. G.D.F. is a full-time employee and shareholder of Weight Watchers International. Weight Watchers has not provided financial support for this study nor did it have any influence on the methods in this study. K.K. is employed by UnitedHealth Group, which supported development of the JOIN for ME curriculum. All other authors declared no conflicts of interest.
References
- 1. Skinner AC, Ravanbakht SN, Skelton JA, et al. Prevalence of obesity and severe obesity in US children, 1999–2016. Pediatrics 2018;141:e20173459. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. French SA, Story M, Perry CL. Self-esteem and obesity in children and adolescents: A literature review. Obes Res 1995;3:479–490 [DOI] [PubMed] [Google Scholar]
- 3. Griffiths LJ, Parsons TJ, Hill AJ. Self-esteem and quality of life in obese children and adolescents: A systematic review. Int J Pediatr Obes 2010;5:282–304 [DOI] [PubMed] [Google Scholar]
- 4. Schwimmer JB, Burwinkle TM, Varni JW. Health-related quality of life of severely obese children and adolescents. JAMA 2003;289:1813–1819 [DOI] [PubMed] [Google Scholar]
- 5. Wardle J, Cooke L. The impact of obesity on psychological well-being. Best Pract Res Clin Endocrinol Metab 2005;19:421–440 [DOI] [PubMed] [Google Scholar]
- 6. Alberga A, Sigal R, Goldfield G, et al. Overweight and obese teenagers: Why is adolescence a critical period. Pediatr Obes 2012;7:261–273 [DOI] [PubMed] [Google Scholar]
- 7. Freedman DS, Khan LK, Serdula MK, et al. Inter-relationship among childhood BMI, childhood height, and adult obesity: The Bogalusa heart study. Int J Obes 2004;28:10–16 [DOI] [PubMed] [Google Scholar]
- 8. Freedman DS, Dietz WH, Tang R, et al. The relation of obesity throughout life to carodit intima-media thickness in adulthood: The Bogalusa heart study. Int J Obes 2004;28:159–166 [DOI] [PubMed] [Google Scholar]
- 9. Whitlock EP, O'Connor EA, Williams SB, et al. Effectiveness of Weight Management Programs in Children and Adolescents. Agency for Healthcare Research and Quality: Rockville, MD, 2008 [Google Scholar]
- 10. O'Connor EA, Evans CV, Burda BU, et al. Screening for obesity and intervention for weight management in children and adolescents: Evidence report and systematic review for the US Preventive Services Task Force. JAMA 2017;317:2427–2444 [DOI] [PubMed] [Google Scholar]
- 11. United States Preventive Services Task Force, Barton M. Screening for obesity in children and adolescents: US Preventive Services Task Force recommendation statement. Pediatrics 2010;125:361–367 [DOI] [PubMed] [Google Scholar]
- 12. Wilfley DE. Improving access and systems of care for evidence-based childhood obesity treatment: Conference key findings and next steps. Obesity (Silver Spring, Md.) 2017;25:16. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Norman G, Huang J, Davila EP, et al. Outcomes of a 1-year randomized controlled trial to evaluate a behavioral ‘stepped-down’ weight loss intervention for adolescent patients with obesity. Pediatr Obes 2016;11:18–25 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Nguyen B, Shrewsbury VA, O'Connor J, et al. Two-year outcomes of an adjunctive telephone coaching and electronic contact intervention for adolescent weight-loss maintenance: The Loozit randomized controlled trial. Int J Obes (Lond) 2013;37:468–472 [DOI] [PubMed] [Google Scholar]
- 15. Love-Osborne K, Fortune R, Sheeder J, et al. School-based health center-based treatment for obese adolescents: Feasibility and body mass index effects. Child Obes (Print) 2014;10:424–431 [DOI] [PubMed] [Google Scholar]
- 16. Pbert L, Druker S, Barton B, et al. A School-Based Program for Overweight and Obese Adolescents: A Randomized Controlled Trial. J Sch Health 2016;86:699–708 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. Foster GD, Sundal D, Lent MR, et al. 18-month outcomes of a community-based treatment for childhood obesity. Pediatr Obes 2014;9:e63–e67 [DOI] [PubMed] [Google Scholar]
- 18. Foster GD, Sundal D, McDermott C, et al. Feasibility and preliminary outcomes of a scalable, community-based treatment of childhood obesity. Pediatrics 2012;130:652–659 [DOI] [PubMed] [Google Scholar]
- 19. Marcia J. Identity in adolescence. In: Adleson J. (ed), Handbook of Adolescent Psychology. Wiley: New York, 1980. [Google Scholar]
- 20. Berk L. Child Development, 4th ed. Allyn and Bacon: Needham Heights, MA, 1997 [Google Scholar]
- 21. Salvy SJ, Roemmich JN, Bowker JC, et al. Effect of peers and friends on youth physical activity and motivation to be physically active. J Pediatr Psychol 2009;34:217–225 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22. Salvy SJ, Bowker JW, Roemmich JN, et al. Peer influence on children's physical activity: An experience sampling study. J Pediatr Psychol 2008;33:39–49 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23. Jelalian E, Sato AF, Hart CN. The effect of group-based weight control intervention on adolescent psychosocial outcomes: Perceived peer rejection, social anxiety and self-concept. Child Health Care 2011;40:197–211 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24. Jelalian E, Hadley W, Sato A, et al. Adolescent weight control: An intervention targeting parent communication and modeling compared with minimal parental involvement. J Pediatr Psychol 2015;40:203. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25. Stout RL, Wirtz PW, Carbonari JP, Del Boca FK. Ensuring balanced distribution of prognostic factors in treatment outcome research. J Stud Alcohol Suppl 1994;12:70–75 [DOI] [PubMed] [Google Scholar]
- 26. Berkowitz R, Wadden TA, Gehrman CA, et al. Meal replacements in the treatment of adolescent obesity: A randomized controlled trial. Obesity 2011;19:1193–1199 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27. Jelalian E, Richardson EE, Mehlenbeck RS, et al. Behavioral weight control treatment combined with supervised exercise or peer enhanced adventure intervention for overweight adolescents. J Pediatri 2010;157:923–928 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28. Naar-King S, Ellis D, Idalski Carcone A, et al. Sequential multiple assignment randomized trial (SMART) to construct weight loss interventions for African American adolescents. J Clin Child Adolesc Psychol 2016;45:428–441 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29. Jelalian E, Foster GD, Sato AF, et al. Treatment adherence and facilitator characteristics in a community based pediatric weight control intervention. Int J Behav Nutr Phys Act 2014;11:17. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30. Centers for Disease Control and Prevention. A SAS Program for the 2000. CDC Growth Charts (ages 0 to <20 years). 2016. Available at https://www.cdc.gov/nccdphp/dnpao/growthcharts/resources/sas.htm Last accessed April3, 2018
- 31. Hunsberger M, O'Malley J, Block T, Norris JC. Relative validation of Block Kids Food Screener for dietary assessment in children and adolescents. Matern Child Nutr 2015;11:260–270 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32. Calabro M, Welk GJ, Eisenmann J. Validation of the SenseWear Pro armband algorithms in children. Med Sci Sports Exerc 2009;41:1714–1720 [DOI] [PubMed] [Google Scholar]
- 33. Calabró MA, Stewart JM, Welk GJ. Validation of pattern-recognition monitors in children using doubly labeled water. Med Sci Sports Exerc 2013;45:1313–1322 [DOI] [PubMed] [Google Scholar]
- 34. Trost SG, Pate RR, Freedson PS, et al. Using objective physical activity measures with youth: How many days of monitoring are needed? Med Sci Sports Exerc 2000;32:426–431 [DOI] [PubMed] [Google Scholar]
- 35. Eys M, Loughead T, Bray SR, Carron AV. Development of a cohesion questionnaire for youth: The Youth Sport Environment Questionnaire. J Sport Exerc Psychol 2009;31:390–408 [DOI] [PubMed] [Google Scholar]
- 36. Wilson D, Kitzman-Ulrich H, Resnicow K, et al. An overview of the Families Improving Together (FIT) for weight loss randomized controlled trial in African American families. Contemp Clin Trials 2015;42:145–157 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37. Jelalian E, Hart CN, Mehlenbeck RS, et al. Predictors of attrition and weight loss in an adolescent weight control program. Obesity 2008;16:1318–1323 [DOI] [PubMed] [Google Scholar]
- 38. Preacher K, Curran P, Bauer D. Computational tools for probing interactions in multiple linear regression, multilevel modeling, and latent curve analysis. J Educ Behav Stat 2006;31:437–448 [Google Scholar]
- 39. Dawson J. Moderation in management research: What, why, when, and how. J Bus Psychol 2014;29:1–13 [Google Scholar]
- 40. Nguyen B, Shrewsbury VA, O'Connor J, et al. Twelve-month outcomes of the loozit randomized controlled trial: A community-based healthy lifestyle program for overweight and obese adolescents. Arch Pediatr Adolesc Med 2012;166:170–177 [DOI] [PubMed] [Google Scholar]
- 41. Alia KA, Wilson DK, McDaniel T, et al. Development of an innovative process evaluation approach for the Families Improving Together (FIT) for weight loss trial in African American adolescents. Eval Program Plann 2015;49:106–116 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42. Nobles J, Radley D, Dimitri P, Sharman K. Psychosocial interventions in the treatment of severe adolescent obesity: The SHINE Program. J Adolesc Health 2016;59:523–529 [DOI] [PubMed] [Google Scholar]
- 43. de Niet J, Timman R, Bauer S, et al. Short message service reduces dropout in childhood obesity treatment: A randomized controlled trial. Health Psychol 2012;31:797–805 [DOI] [PubMed] [Google Scholar]
- 44. Hingle M, Nichter M, Medeiros M, Grace S. Texting for health: The use of participatory methods to develop healthy lifestyle messages for teens. J Nutr Educ Behav 2013;45:12–19 [DOI] [PubMed] [Google Scholar]
- 45. Woolford SJ, Clark SJ, Strecher VJ, Resnicow K. Tailored mobile phone text messages as an adjunct to obesity treatment for adolescents. J Telemed Telecare 2010;16:458–461 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46. Kolodziejczyk JK, Merchant G, Norman GJ. Reliability and validity of child/adolescent food frequency questionnaires that assess foods and/or food groups. J Pediatr Gastroenterol Nutr 2012;55:4–13 [DOI] [PubMed] [Google Scholar]
- 47. U.S. Department of Health and Human Services, U.S. Department of Agriculture. 2015–2020 Dietary Guidelines for Americans, 8th ed, 2015. https://health.gov/dietaryguidelines/2015/resources/2015-2020_Dietary_Guidelines.pdf (last assessed February19, 2019)
- 48. Kahan B, Morris T. Assessing potential sources of clustering in individually randomised trials. BMC Med Res Methodol 2013;13:58. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 49. Babinski D, Sibley M, Ross J, Pelham W. The effects of single versus mixed gender treatment for adolescent girls with ADHD. J Clin Child Adolesc Psychol 2013;42:243–250 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 50. Chaplin T, Gillham J, Reivich K. Depression prevention for early adolescent girls: A pilot study of all girls versus co-ed groups. J Early Adolesc 2006;26:110–126 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 51. Goldfield G, Mallory R, Prud'homme D, Adamo K. Gender differences in response to a physical activity intervention in overweight and obese children. J Phys Act Health 2008;5:592–606 [DOI] [PubMed] [Google Scholar]
- 52. Rancourt D, Barker D, Jelalian E. Sex as a moderator of adolescents' weight loss treatment outcomes. J Adol Health 2018;62:591–597 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 53. Fitzgibbon ML, Stolley MR, Schiffer L, et al. Obesity reduction black intervention trial (ORBIT): 18-month results. Obesity (Silver Spring, Md.) 2010;18:2317–2325 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 54. Proctor E, Silmere H, Raghavan R, et al. Outcomes for implementation research: Conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health 2011;38:65–76 [DOI] [PMC free article] [PubMed] [Google Scholar]


