Key Points
Question
Does an internet-based weight loss program promote long-term weight loss in low-income postpartum women in the Special Supplemental Nutrition Program for Women, Infants, and Children (WIC program)?
Findings
In this cluster randomized trial including 371 low-income postpartum women, an internet-based program plus the WIC program produced significantly greater weight loss over 12 months compared with the WIC program alone (3.2 kg vs 0.9 kg).
Meaning
Among low-income postpartum women, an internet-based weight loss program plus the WIC program compared with the WIC program alone resulted in significantly greater weight loss over 12 months. Future research is needed to determine cost-effectiveness.
Abstract
Importance
Postpartum weight retention increases lifetime risk of obesity and related morbidity. Few effective interventions exist for multicultural, low-income women.
Objective
To test whether an internet-based weight loss program in addition to the Special Supplemental Nutrition Program for Women, Infants, and Children (WIC program) for low-income postpartum women could produce greater weight loss than the WIC program alone over 12 months.
Design, Setting, and Participants
A 12-month, cluster randomized, assessor-blind, clinical trial enrolling 371 adult postpartum women at 12 clinics in WIC programs from the California central coast between July 2011 and May 2015 with data collection completed in May 2016.
Interventions
Clinics were randomized to the WIC program (standard care group) or the WIC program plus a 12-month primarily internet-based weight loss program (intervention group), including a website with weekly lessons, web diary, instructional videos, computerized feedback, text messages, and monthly face-to-face groups at the WIC clinics.
Main Outcomes and Measures
The primary outcome was weight change over 12 months, based on measurements at baseline, 6 months, and 12 months. Secondary outcomes included proportion returning to preconception weight and changes in physical activity and diet.
Results
Participants included 371 women (mean age, 28.1 years; Hispanic, 81.6%; mean weight above prepregnancy weight, 7.8 kg; mean months post partum, 5.2 months) randomized to the intervention group (n = 174) or standard care group (n = 197); 89.2% of participants completed the study. The intervention group produced greater mean 12-month weight loss compared with the standard care group (3.2 kg in the intervention group vs 0.9 kg in standard care group, P < .001; difference, 2.3 kg (95% CI, 1.1 to 3.5). More participants in the intervention group than the standard care group returned to preconception weight by 12 months (32.8% in the intervention group vs 18.6% in the standard care group, P < .001; difference, 14.2 percentage points [95% CI, 4.7 to 23.5]). The intervention group and standard care group did not significantly differ in 12-month changes in physical activity (mean [95% CI]: −7.8 min/d [−16.1 to 0.4] in the intervention group vs −7.2 min/d [−14.6 to 0.3] in the standard care group; difference, −0.7 min/d [95% CI, −42.0 to 10.6], P = .76), calorie intake (mean [95% CI]: −298 kcal/d [−423 to −174] in the intervention group vs −144 kcal/d [−257 to −32] in the standard care group; difference, −154 kcal/d [−325 to 17], P = .06), or incidences of injury (16 in the intervention group vs 16 in the standard care group) or low breastmilk supply from baseline to month 6 (21 of 61 participants in the intervention group vs 23 of 72 participants in the standard care group) and from month 6 to 12 (13 of 32 participants in the intervention group vs 14 of 37 participants in the standard care group).
Results
Participants included 371 women (mean age, 28.1 years; Hispanic, 81.6%; mean weight above prepregnancy weight, 7.8 kg; mean months post partum, 5.2 months) randomized to the intervention group or standard care group; 89.2% of participants completed the study. The intervention group produced greater mean 12-month weight loss compared with the standard care group. More participants in the intervention group than the standard care group returned to preconception weight by 12 months. The intervention group and standard care group did not significantly differ in 12-month changes in physical activity, calorie intake, or incidences of injury or low breastmilk supply from baseline to month 6 (21 of 61 participants in the intervention group vs 23 of 72 participants in the standard care group) and from month 6 to 12 (13 of 32 participants in the intervention group vs 14 of 37 participants in the standard care group).
12-mo Outcomes | Standard Care Group (n=197) | Intervention Group (n=174) | Between-Group Difference (95% CI) | P Value |
---|---|---|---|---|
Mean weight change, kg | −0.9 (−1.7 to −0.1) | −3.2 (−4.1 to −2.4) | 2.3 (1.1 to 3.5) | <.001 |
At or below preconception weight, No. (%) | 36 (18.6) | 57 (32.8) | 14.2 (4.7 to 23.5) | <.001 |
Change in physical activity, mean (95% CI), min/d | −7.2 (−14.6 to 0.3) |
−7.8 (−16.1 to 0.4) |
−0.7 (−42.0 to 10.6) |
.76 |
Change in calorie intake, mean (95% CI), kcal/d | −144 (−257 to −32) |
−298 (−423 to −174) |
−154 (−325 to 17) |
.06 |
Conclusions and Relevance
Among low-income postpartum women, an internet-based weight loss program in addition to the Special Supplemental Nutrition Program for Women, Infants, and Children (WIC program) compared with the WIC program alone resulted in a statistically significant greater weight loss over 12 months. Further research is needed to determine program and cost-effectiveness as part of the WIC program.
Trial Registration
clinicaltrials.gov Identifier: NCT01408147
This cluster randomized clinical trial assessed the effects of adding an internet-based weight loss intervention program to the Special Supplemental Nutrition Program for Women, Infants, and Children (WIC program) on weight loss over 12 months among low-income, postpartum women.
Introduction
Of the approximately 4 million women who give birth in the United States each year, between 2004 and 2008 an estimated 25% experienced major weight retention after pregnancy, retaining more than 4.5 kg and gaining additional weight during the postpartum year. Prevalence rates of postpartum weight retention are higher among low-income Hispanic women and women with food insecurity, affecting 40% to 60% of this group. Postpartum weight retention tends to be centrally deposited and is an independent risk factor for long-term health consequences for the mother, including increased risk of lifetime obesity, cardiovascular disease, and type 2 diabetes. Also, women with high postpartum weight retention are heavier prior to their next pregnancy, which increases risk of pregnancy-related complications, obesity, and health complications in future offspring.
Several researchers and government agencies have called for empirical studies evaluating interventions that occur after pregnancy with the specific aim of reducing high postpartum weight retention and the health risks of maternal postpartum obesity. Intervention trials testing methods to reduce postpartum weight retention in women have been limited by short duration (≤6 months), high attrition (30%-40%), and lack of efficacy, particularly among low-income or racial/ethnic minorities. Internet-based interventions have been shown to be effective, although less data are available from community settings with lower income and minority samples.
The primary goal of this cluster randomized trial was to evaluate the 12-month efficacy of a culturally and linguistically adapted, primarily internet-based behavioral weight loss intervention for low-income postpartum women in the Special Supplemental Nutrition Program for Women, Infants, and Children (WIC program).
Methods
Design
Fit Moms/Mamás Activas was a cluster randomized clinical trial. The trial protocol and statistical analysis are available in the Supplement and have been published previously.
Participants
Procedures were approved by the California Polytechnic State University institutional review board, and all participants provided written informed consent. Recruitment occurred between July 2011 and May 2015 across 12 WIC clinics in Santa Barbara (n = 6), San Luis Obispo (n = 4), and Ventura (n = 2) counties of the California central coast, which were selected based on size, proximity to the university, and representativeness. Clinics had to accept the randomization of the study, recruitment, and intervention protocols. Participant eligibility was based on self-report and included being 6 weeks to 12 months post partum and having a body mass index (BMI; calculated as weight in kilograms divided by height in meters squared) of more than 25 or a BMI from 22 to 24.9 but exceeding prepregnancy weight by 4.5 kg or more. Participants were aged 18 to 40 years, spoke English or Spanish, were nonsmoking, owned a cell phone, and had a fifth-grade education or higher. Exclusion criteria are displayed in Figure 1.
Interventions
The study statistician randomized the 12 clinics to the 2 conditions, blocking on county (ie, San Luis Obispo [4 clinics]; Santa Barbara [6 clinics]; Ventura [2 clinics]) and using the R (R Foundation), version 3.3.1, statistical software package.
Standard Care Group
The standard care group received all aspects of the standard WIC program plus a brief orientation to the study and newsletters every 2 months with information about weight control, exercise, nutrition, and wellness.
Intervention Group
The intervention group received all elements of the standard WIC program plus a 12-month primarily internet-based weight loss program adapted from prior programs. Calorie goals (from 1200 to 1800 calories per day with 300 additional calories for mothers who were breastfeeding) were provided based on study entry weight and breastfeeding status. Physical activity goals gradually increased to 30 minutes or more per day on most days of the week. The program was available in English and Spanish and provided guidance and resources, automated feedback, weekly lessons, a web diary, a weight and physical activity tracker, instructional and inspirational videos, and a message board. Four weekly text messages notified participants of new website content and provided motivation, support, and feedback. Study interventionists held monthly face-to-face group sessions at WIC clinics. WIC program staff were encouraged to reinforce website use via distribution of promotional cards during participants’ regular WIC program visits.
Outcome Assessments
Assessments were conducted at study entry, 6 months, and 12 months. Participants received $25 for completing baseline and 6-month assessments and $50 for the 12-month assessment. Assessment staff was blinded to randomization. In a few instances, the assessor became unblinded and in these cases was reassigned and did not complete any unblinded assessments.
The primary outcome was weight change over 12 months, based on measured weights at baseline, 6 months, and 12 months. Secondary outcomes were proportion of women returning to preconception weight and changes in physical activity and dietary intake. Other secondary outcomes are not reported in this article, including weight control behaviors and psychosocial factors. Outcomes that were not prespecified included waist circumference, percentage weight loss, and proportions achieving 5% or more and 10% or more weight loss. Post hoc analyses estimated program costs from a payer perspective.
Weight was assessed in lightweight clothing using a calibrated scale. Height was measured without shoes. Prepregnancy weight was based on self-report at the time of the last menstrual period. Waist circumference was measured at the midpoint between the highest point of the iliac crest and the lowest point of the costal margin using a tape measure. Race and ethnicity, education, income, employment status, and parity were assessed as potential intervention moderators and measured using self-report questionnaires with fixed categories. Adverse events were based on participant report of the number of injuries associated with physical activity, new medical diagnoses, overnight hospitalizations, and experiences of low breastmilk supply.
Dietary intake was assessed using 24-hour recalls on 2 random days over a week and completed using the National Cancer Institute Automated Self-Administered 24-Hour (ASA24) dietary assessment tool. A waist-worn accelerometer (GT3X+; ActiGraph) measured physical activity. Wear time was determined using the Choi algorithm. For inclusion in the analysis, the accelerometer had to be worn 8 hours or longer per day and worn for at least 2 days but no more than 7 days.
In exploratory analysis, costs were estimated from the payer perspective and excluded 1-time “sunk costs” (eg, intervention, equipment costs, staff training). Mean labor costs for intervention delivery (ie, monthly 60-minute group visits; 15 minutes per week message board monitoring) were estimated at 12.16 hours per participant × the mean hourly rate of a community health worker ($17.95 per hour). Labor costs included having a computer scientist troubleshoot technical issues (15 minutes per participant × $40.95 per hour). Nonlabor costs (eg, paper materials, scales, pedometers) were estimated at $48.00 per participant.
Statistical Analysis
Sample Size
The targeted sample size of 12 clinics with 34 participants per clinic (N = 408) was projected to provide 80% or more power to detect a clinically important difference of 2 kg or more in weight change between the 2 groups over 12 months with an SD of 4.5 kg and an intraclass correlation of 0.02, using a 2-sided t test with a significance level of .05. The minimal clinically important difference of 2 kg was selected based on prior research showing that 1 kg or more of postpartum weight retention and modest gains of 0.8 kg to 1 kg per year in young adults have been linked with metabolic disease; also, modest 2% to 5% weight losses have been shown to have metabolic health benefits. The sample size calculations accommodated the possibility of 2 clinic withdrawals (ie, potential analytic sample of 10 clinics), and an overall participant attrition of up to 30% (122 of 407 participants) and intention-to-treat analyses.
Statistical Analysis Plan
Missing data were assumed to be missing at random. The approach included a likelihood-based, linear mixed-effects model that included the randomized clinics (and excluded 1 clinic that withdrew) and all participants in these clinics as random effects. This model allowed for participants to have partial missing response data and still be included in the model without imputation. Also, a sensitivity analysis was conducted for the primary outcome in which 10 random multivariate normal imputations based on the observed variables (group, ethnicity, education, income, employment status, waist circumference, and weight) at previous assessments were used to impute missing weight data. Model parameter estimates from the imputed data sets were pooled, compared with estimates from the primary analysis, and verified to be within 2 standard errors of each other. The t tests or χ2 tests were used to compare women with complete and incomplete records on group assignment, maternal BMI, ethnicity, weeks post partum, income, education, and age.
To test the primary hypothesis, a generalized linear mixed-effect model was used (intervention group as a fixed effect and clinic and subject as random effects). A group × time interaction term (fixed effect) tested whether the change in weight over time differed significantly. The model included participant-level covariates (ie, ethnicity, weeks post partum at study entry, lactation, and age). Partial F tests were first used to simultaneously test all main effects and interactions. If the group × time interaction was statistically significant (P < .005), the equality of mean changes in the 2 groups at each intermediate time point was tested. Similar linear mixed-effects models with the same participant covariates were conducted to examine changes over time in waist circumference, percentage of weight lost, and generalized linear mixed-effects models with binomial errors and logit link were used to examine effects on proportions achieving prepregnancy weight or below, and proportions achieving 5% or more and 10% or more reduction in initial body weight. Unadjusted P values were reported, but a Bonferroni correction factor (P < .005) based on 10 comparisons was used for interpretation of significance (family-wise error rate = 0.05). Linear mixed-effects models were also used to examine whether baseline variables (ethnicity, education, employment, lactation, BMI category at study entry, income, age, weeks post partum at entry, and parity) moderated weight loss outcomes. In all analyses, model conditions for numeric responses were verified with diagnostic plots. Partial correlation analysis examined the relationship between engagement variables (eg, login frequency, attendance) and weight change, adjusting for the same covariates. R (R Foundation), version 3.3.1; SPSS (IBM), version 23; and JMP (SAS Institute), version 12.2.0, statistical software packages were used for all analyses.
Results
Figure 1 summarizes the flow of participants through the study. Of the 12 WIC clinics approached for participation, all agreed to participate and were randomized at study onset. Prior to initiation of participant recruitment but after randomization, 1 intervention clinic withdrew participation due to declines in staff, space, and participant census, leaving an expected sample size of 374. The target sample size of 34 participants was met in 8 clinics, exceeded in 2 clinics (n = 35 and n = 37), not met in 1 clinic (n = 27), reflecting a total enrollment of 371. Participants were a mean age of 28 years, predominantly Hispanic (81.6%), a mean 7.8 kg above prepregnancy weight, and a mean 5.2 months post partum; the 2 study groups did not significantly differ on baseline measures (Table 1). Participant retention was 92.7% at 6 months and 89.2% at 12 months. The demographic characteristics did not significantly differ between participants who attended vs did not attend the 12-month visit.
Table 1. Baseline Characteristics of Postpartum Participants in the Standard Care vs Intervention Groups.
Characteristic | Total (N = 370) |
Standard Care Group (n = 196)a |
Intervention Group (n = 174)a |
---|---|---|---|
Age, mean (SD), y | 28.1 (5.4) | 28.6 (5.5) | 27.5 (5.2) |
Hispanic/Latino, No. (%) | |||
Yes | 302 (81.6) | 156 (79.6) | 146 (83.9) |
No | 68 (18.3) | 40 (20.4) | 28 (16.1) |
Marital status, No. (%)b | |||
Married or with significant other | 285 (77.9) | 145 (74.7) | 140 (81.4) |
Divorced | 19 (5.2) | 13 (6.7) | 6 (3.5) |
Widowed or never married | 62 (16.9) | 36 (18.6) | 26 (15.1) |
Education, No. (%)b | |||
Grade school or junior high | 80 (21.7) | 43 (22.1) | 37 (21.3) |
High school | 136 (36.9) | 71 (36.4) | 65 (37.4) |
Some college and college | 153 (41.5) | 81 (41.5) | 72 (41.4) |
Employment, No. (%)b | |||
Unemployed | 273 (74.0) | 150 (76.8) | 123 (70.7) |
Homemaker | 169 | 94 | 75 |
Student | 20 | 12 | 8 |
Maternity leave | 41 | 23 | 18 |
Other | 43 | 21 | 22 |
Employed | 96 (26.0) | 45 (23.1) | 51 (29.3) |
Clerical | 30 | 14 | 16 |
Trade and factory | 22 | 12 | 10 |
Professional | 12 | 6 | 6 |
Other | 32 | 13 | 19 |
Annual household income, No. (%), $b | |||
<10 000 | 69 (18.8) | 40 (20.6) | 29 (16.7) |
10 000-19 999 | 114 (31.0) | 61 (31.4) | 53 (30.5) |
20 000-29 999 | 107 (29.1) | 51 (26.3) | 56 (32.2) |
≥30 000 | 78 (21.2) | 42 (21.6) | 36 (20.7) |
Childbearing history, No. (%) | |||
Primiparous | 95 (25.7) | 51 (26.0) | 44 (25.3) |
Multiparousc | 275 (74.3) | 145 (74.0) | 130 (74.7) |
Months post partum, mean (SD) | 5.2 (3.2) | 5.0 (3.2) | 5.6 (3.1) |
Currently breastfeeding, No. (%) | 231 (62.4) | 110 (63.2) | 121 (61.7) |
Waist circumference, mean (SD), cm | 98.4 (11.5) | 98.0 (11.6) | 98.9 (11.4) |
Weight, mean (SD), kg | 79.9 (5.6) | 79.8 (16.3) | 80.1 (14.8) |
BMI, mean (SD) | 31.7 (5.1) | 31.6 (5.2) | 31.9 (5.0) |
At study entry, No. (%) | |||
22-24.9d | 13 (3.5) | 7 (3.6) | 6 (3.4) |
25.0-29.9 | 146 (39.6) | 82 (42.1) | 64 (36.8) |
≥30 | 210 (56.9) | 106 (54.4) | 104 (59.8) |
Prepregnancy, No. (%)b,e | |||
≤24.9 | 97 (26.5) | 52 (26.9) | 45 (25.9) |
25.0-29.9 | 141 (38.4) | 74 (38.3) | 67 (38.5) |
≥30 | 129 (35.1) | 67 (34.7) | 62 (35.6) |
PPWR, mean (SD) kg | 7.8 (7.4) | 7.4 (7.3) | 8.2 (7.4) |
≤5 kg above prepregnancy weight, No, (%)b | 117 (32.0) | 50 (25.9) | 67 (38.5) |
Abbreviations: BMI, body mass index (calculated as weight in kilograms divided by height in meters squared); PPWR; postpartum weight retention (calculated as the difference between self-reported prepregnancy weight and weight measured at study entry).
The standard care group received the standard Special Supplemental Nutrition Program for Women, Infants, and Children (WIC program). The intervention group received the standard WIC program plus a 12-month primarily internet-based weight loss program adapted from prior programs.
Some participants declined to answer the questions regarding marital status (standard care, 2; intervention, 2), education (standard care, 2), employment (standard care, 1), income (standard care, 2), and preconception weight (standard care, 3).
Seven participants reported having twins during their most recent pregnancy.
For inclusion, participants could have a BMI of 25 or more or could have a BMI from 22 to 24.9 but had to exceed their prepregnancy weight by 4.5 kg or more.
BMI status before pregnancy was based on self-reported preconception weight and measured height at study entry.
Primary Outcome
Weight change variables are summarized in Table 2. A significant interaction between group and time was observed, with greater weight loss in the intervention group than the standard care group (P < .001; Figure 2). Over 12 months, the standard care group lost an average of 0.9 kg, whereas the intervention group lost 3.2 kg. Weight loss was 2.3 kg (95% CI, 1.1 to 3.5) greater in the intervention group compared with the standard care group over 12 months (P < .001). Results from the sensitivity analysis using multiple imputations were similar with a weight loss at 12 months of 2.9 kg in the intervention group and 1.3 kg in the standard care group, representing a difference of 1.6 kg (95% CI, 0.1 to 3.0; P = .03). Several demographic factors were examined collectively as potential treatment effect modifiers. No significant moderators emerged (F13,332 = 0.75; P = .71).
Table 2. Change in Weight, Waist Circumference, and Percentage of Weight Lost for Postpartum Participants in the Standard Care vs Intervention Groups.
Standard Care Group (n = 193)a |
Intervention Group (n = 174)a |
Between-Group Differences (95% CI) |
P Value | Model P Valuesb | |||
---|---|---|---|---|---|---|---|
Groupc | Timed | Group × Timee | |||||
Primary Outcome, Least-Squares Mean (95% CI)f | |||||||
Weight, kg | .63 | <.001 | <.001 | ||||
Baseline | 82.4 (77.9 to 87.1) |
82.5 (77.5 to 87.5) |
|||||
Change from baseline to 6 mo | −1.0 (−1.8 to −0.2) |
−3.1 (−4.0 to −2.3) |
2.1 (0.9 to 3.3) |
<.001 | |||
Change from baseline to 12 mo | −0.9 (−1.7 to −0.1) |
−3.2 (−4.1 to −2.4) |
2.3 (1.1 to 3.5) |
<.001 | |||
Outcomes Not Prespecified, Least-Squares Mean (95% CI)f | |||||||
Waist circumference, cm | .63 | <.001 | <.001 | ||||
Baseline | 98.8 (96.0 to 101.7) |
99.7 (96.5 to 102.9) |
|||||
Change from baseline to 6 mo | −1.0 (−2.0 to 0.01) |
−4.0 (−5.1 to −3.0) |
3.0 (1.5 to 4.5) |
<.001 | |||
Change from baseline to 12 mo | −1.2 (−2.2 to −0.2) |
−4.0 (−5.1 to −2.9) |
2.8 (1.3 to 4.3) |
<.001 | |||
Weight change from baseline, % | .003 | .96 | .75 | ||||
Change from baseline to 6 mo | −2.0 (−3.3 to −0.8) |
−4.9 (−6.2 to −3.5) |
2.8 (1.3 to 4.3) |
<.003 | |||
Change from baseline to 12 mo | −1.9 (−3.13 to −0.7) |
−5.0 (−6.3 to −3.7) |
3.1 (1.6 to 4.6) |
<.002 | |||
Proportion achieving weight loss outcomes from baseline to 12 mog | Between-Group Difference, No. of Participants (% [95% CI]) | ||||||
≥5% Weight loss, No. (%)g | 63 (31.9) | 85 (48.7) | 22 (16.8 [6.3 to 27.1]) | .005g | |||
≥10% Weight loss, No. (%)g | 25 (12.8) | 45 (26.0) | 20 (13.2 [4.6 to 21.7]) | .007g |
The standard care group received the standard Special Supplemental Nutrition Program for Women, Infants, and Children (WIC program). The intervention group received standard WIC program plus a 12-month primarily internet-based weight loss program adapted from prior programs.
Based on linear mixed-effects model with covariates (ie, ethnicity, weeks post partum at study entry, lactation, and age). Three women from the standard care group were missing 1 or more covariate measurements at study entry and were excluded from analysis.
Indicates whether there were any significant group differences in the outcome measure overall, regardless of time.
Indicates whether the outcome changed over time, regardless of group.
Indicates group differences in the outcome over time. Model validation results: for weight, partial F test P < .001, R2 = 0.96; for waist circumference, partial F test P < .001, R2 = 0.91; for percent weight change, partial F test P = .002, R2 = 0.76.
Presented as least-squares mean (95% CI), which is the model-estimated response under each treatment condition evaluated for covariates (ie, ethnicity, weeks post partum at study entry, lactation, and age) at their mean levels.
Predicted counts based on generalized linear model with covariates (ie, ethnicity, weeks post partum at study entry, lactation, and age); the resulting odds ratio for group differences in 5% or more weight loss was 2.0 (95% CI, 1.2-3.3) and for 10% or more weight loss was 2.4 (95% CI, 1.3-4.6). Raw (unadjusted) counts for proportions reaching 5% or more weight loss at 12 months were 56 of 172 participants in the standard care group and 67 of 152 in the intervention group; for 10% or more weight loss, 23 of 172 participants for the standard care group and 35 of 152 for the intervention group.
Secondary Outcomes
The intervention increased the percentage of participants who achieved preconception weight or below (P < .001) (Table 3). Overall, 32.8% of the intervention group and 18.6% of the standard care group were at preconception weight or below by 12 months, representing a difference of 14.2 percentage points (95% CI, 4.7 to 23.5; P < .001). The intervention had no significant (P = .76) effects on moderate to vigorous physical activity (difference, −0.7 min/d [95% CI, −42.0 to 10.6]) or other activity parameters, which declined over time (Table 4). Calorie intake declined over time (Table 4), but there were no significant (P = .06) group differences (difference, −154 kcal/d [95% CI, −325 to 17]).
Table 3. Proportions at or Below Preconception Weight for Postpartum Participants in the Standard Care vs Intervention Groupsa.
No. of Participants/Total Participants (%) | Between-Group Differences, No. of Participants (% [95% CI])c,d | P Value | ||
---|---|---|---|---|
Standard Care Groupb | Intervention Groupb | |||
Baseline | ||||
Adjustedc,d | 28/193 (14.4) | 15/174 (8.5) | ||
Unadjusted | 32/193 (16.6) | 16/174 (9.2) | ||
6 mo | ||||
Adjustedd | 35/193 (18.3) | 51/174 (29.2) | 16 (10.9 [2.0-20.4]) | <.001 |
Unadjusted | 37/179 (20.6) | 50/161 (31.7) | ||
12 mo | ||||
Adjustedd | 36/193 (18.6) | 57/174 (32.8) | 21 (14.2 [4.7-23.5]) | <.001 |
Unadjusted | 36/172 (20.9) | 51/152 (35.6) |
Preconception weight was defined as self-reported prepregnancy weight plus 0.9 kg.
The standard care group received the standard Special Supplemental Nutrition Program for Women, Infants, and Children (WIC program). The intervention group received the standard WIC program plus a 12-month primarily internet-based weight loss program adapted from prior programs.
Three women were excluded from the standard care group due to missing covariates. Data reflect No. of Participants/Total No. (%) of women who, at study enrollment, were at or below their self-reported prepregnancy weight plus 0.9 kg. At enrollment, women were a mean 5.2 months post partum.
Predicted counts were based on a generalized linear model with covariates (ie, ethnicity, weeks post partum at study entry, lactation, and age); the resulting odds ratio for group differences at 6 months was 1.9 (95% CI, 1.4-2.4) and at 12 months was 2.1 (95% CI, 1.6-2.6). Only the adjusted values were analyzed for between-group difference.
Table 4. Change in Physical Activity and Diet for Postpartum Participants in the Standard Care vs Intervention Groupsa,b.
Standard Care Groupc | Intervention Groupc | Between-Group Differences, Mean (95% CI) |
P Value | Model P Valuesd | |||||
---|---|---|---|---|---|---|---|---|---|
Least-Squares Mean (95% CI)h | No. of Participants | Least-Squares Mean (95% CI)h | No. of Participants | Groupe | Timef | Group × Timeg | |||
Sedentary, min/d | .88 | <.001 | .46 | ||||||
Baseline | 319 (288 to 350) | 160 | 298 (264 to 332) | 136 | |||||
Change from baseline to 6 mo | −49 (−84 to −14) | 103 | −27 (−65 to 12) | 92 | 23 (−30 to 75) | .40 | |||
Change from baseline to 12 mo | −75 (−112 to −37) | 85 | −42 (−83 to 0) | 70 | 33 (−24 to 90) | .24 | |||
Light-intensity physical activity (1.5 to <3.0 METs), min/d | .47 | <.001 | .71 | ||||||
Baseline | 261 (229 to 294) | 160 | 266 (231 to 302) | 136 | |||||
Change from baseline to 6 mo | −66 (−98 to −34) | 103 | −46 (−81 to −11) | 92 | 20 (−28 to 68) | .41 | |||
Change from baseline to 12 mo | −77 (−111 to −42) | 85 | −70 (−108 to −32) | 70 | 7 (−45 to 59) | .80 | |||
Total moderate to vigorous physical activity (≥3 METs), min/d | .61 | .03 | .76 | ||||||
Baseline | 37.7 (30.7 to 44.6) | 160 | 38.9 (31.3 to 46.4) | 136 | |||||
Change from baseline to 6 mo | −5.6 (−12.6 to 1.3) | 103 | −2.3 (−9.9 to 5.2) | 92 | 3.3 (−7.1 to 13.7) | .53 | |||
Change from baseline to 12 mo | −7.2 (−14.6 to 0.3) | 85 | −7.8 (−16.1 to 0.4) | 70 | −0.7 (−42.0 to 10.6) | .91 | |||
Total calories, kcal/dh | .02 | <.001 | .06 | ||||||
Baseline | 1785 (1691 to 1879) | 191 | 1762 (1661 to 1863) | 170 | |||||
Change from baseline to 6 mo | −123 (−232 to −13) | 154 | −299 (−418 to −181) | 129 | −176 (−340 to −12) | .03 | |||
Change from baseline to 12 mo | −144 (−257 to −32) | 143 | −298 (−423 to −174) | 111 | −154 (−325 to 17) | .07 | |||
Percentage of calories from carbohydrates | .01 | .71 | .46 | ||||||
Baseline | 48.4 (46.7 to 50.1) | 192 | 50.0 (48.1 to 51.9) | 170 | |||||
Change from baseline to 6 mo | −0.9 (−2.8 to 1.0) | 154 | 0.3 (−1.9 to 2.4) | 129 | 1.1 (−4.1 to 1.8) | .44 | |||
Change from baseline to 12 mo | −1.5 (−3.5 to 0.5) | 143 | 0.3 (−1.9 to 2.5) | 111 | 1.8 (−4.9 to −1.2) | .23 | |||
Percentage of calories from protein | .88 | .53 | .49 | ||||||
Baseline | 17.1 (16.2 to 18.1) | 192 | 17.2 (16.2 to 18.1) | 170 | |||||
Change from baseline to 6 mo | 0.0 (−0.9 to 0.93) | 154 | 0.5 (−0.5 to 1.6) | 129 | −0.5 (−1.9 to 0.9) | .44 | |||
Change from baseline to 12 mo | 0.6 (−0.4 to 1.51) | 143 | 0.2 (−0.9 to 1.3) | 111 | 0.4 (−1.0 to 1.8) | .63 | |||
Percentage of calories from fat | .02 | .91 | .52 | ||||||
Baseline | 35.2 (33.7 to 36.6) | 192 | 33.4 (31.8 to 34.9) | 170 | |||||
Change from baseline to 6 mo | 0.8 (−0.74 to 2.3) | 154 | −0.5 (−2.1 to 1.2) | 129 | 1.2 (−1.1 to 3.5) | .27 | |||
Change from baseline to 12 mo | 0.4 (−1.2 to 1.9) | 143 | 0.2 (−1.6 to 1.9) | 111 | 0.2 (−2.2 to 2.6) | .86 |
Abbreviation: MET, metabolic equivalent.
Physical activity was based on objective assessment for a 1-week period; 97.5% of participants (361 of 370) completed this at baseline, 71.0% (263 of 370) at 6 months, and 60.5% (224 of 370) at 12 months. Of these, proportions with valid data were 296 of 361 participants (82%) at baseline; 195 of 263 participants (74%) at 6 months and 155 of 224 participants (69%) at 12 months. The analysis excluded 28 women (16 from the standard care group and 12 from the intervention group) due to suboptimal actigraph wear time at all 3 assessments (ie, <8 h and <2 d) and 3 from the standard care group due to missing covariates, resulting in 339 included in the analytic sample. Participants who completed vs did not complete the actigraph measures at 12 months did not significantly differ in group assignment, age, ethnicity, weight status, weeks post partum, income, or employment at study entry.
Diet was based on data from the Automated Self-Administered 24-Hour dietary assessment tool; 99.2% of participants (367 of 370) completed this assessment at baseline, 78.1% (289 of 370) at 6 months, and 69.7% (258 of 370) at 12 months. The analysis excluded 3 women from the standard care group due to missing covariates, resulting in 367 included in the analytic sample. Participants who completed vs did not complete the diet measure at 12 months did not significantly differ in group assignment, age, ethnicity, weight status, weeks post partum, income, or employment at study entry.
The standard care group received the standard Special Supplemental Nutrition Program for Women, Infants, and Children (WIC program). The intervention group received the standard WIC program plus a 12-month primarily internet-based weight loss program adapted from prior programs.
Mixed-effects models with covariates (ethnicity, weeks post partum at study entry, weight status, lactation, and age).
Indicates group differences in the outcome measure overall, regardless of time.
Indicates outcome changes over time, regardless of group.
Indicates group differences in the outcome over time.
Presented as least-squares mean (95% CI), which is the model-estimated response under each treatment condition evaluated for covariates (ie, ethnicity, weight status, weeks post partum at study entry, lactation, and age) at their mean levels.
Outcomes Not Prespecified
As shown in Table 2, at 12 months, waist circumference was lower in the intervention group compared with the standard care group (difference, 2.8 cm [95% CI, 1.3 to 4.3]; P < .001). Also, the percentage of weight loss over 12 months was higher in the intervention group than the standard care group (difference, 3.1 percentage points [95% CI, 1.6 to 4.6]; P < .002) (Table 2). The proportion reaching 5% or more weight loss was higher in the intervention group than in the standard care group at 12 months (difference, 16.8 percentage points [95% CI, 6.3 to 27.1]; P < .005), as was the proportion reaching 10% or more weight loss (difference, 13.2 percentage points [95% CI, 4.6 to 21.7]; P = .007).
Post Hoc Outcomes
Estimated mean labor costs for intervention delivery ($218.27) and technical assistance ($10.23 per participant) totaled $228.50 per participant annually, and nonlabor costs (eg, paper materials, scales, pedometers) were approximately $48.00 per participant. Thus, total per-participant costs from the payer perspective were estimated at $276.00 per participant annually.
Intervention Adherence
The mean number of logins was 74.0 (SD, 111.0) overall and 6.0 (SD, 9.3) logins per month; these means included the 12 participants who never logged onto the website. Login frequency was significantly related to 12-month weight loss (R = 0.25; P = .003). Attendance at monthly group meetings (range, 0-12 meetings) was a mean 4.4 (SD, 2.7) visits overall, representing 37% of expected visit attendance. Attendance at group visits was also significantly correlated with greater 12-month weight loss (R = 0.18; P = .03).
Adverse Events
No important group differences were observed in safety alerts or adverse events (Table 5).
Table 5. Adverse Events for the Standard Care vs Intervention Groups (N = 370).
Standard Care Group (n = 196)a |
Intervention Group (n = 174)a |
|
---|---|---|
Injury From Physical Activity, No. of Participants | ||
From baseline to 6 mo | 8 | 11 |
From 6 mo to 12 mo | 7 | 5 |
Overnight Hospitalization, No. of Participants | ||
From baseline to 6 mo | 2 | 2 |
From 6 mo to 12 mo | 5 | 5 |
New Medical Diagnosis or Treatment, No. of Participants | ||
From baseline to 6 mo | 12 | 16 |
From 6 mo to 12 mo | 17 | 17 |
If Breastfeeding, Current Breastmilk Supply Less Than Needed, No. of Participants/Total Participants (%) | ||
From baseline to 6 mo | 23/72 (32) | 21/61 (34) |
From 6 mo to 12 mo | 14/37 (38) | 13/32 (41) |
The standard care group received the standard Special Supplemental Nutrition Program for Women, Infants, and Children (WIC program). The intervention group received the standard WIC program plus a 12-month primarily internet-based weight loss program adapted from prior programs.
Discussion
This cluster randomized trial found that an internet-based weight loss program integrated into the WIC program with monthly group visits was effective in promoting significant weight loss in low-income postpartum women. The intervention was effective across all demographic characteristics and among women who were breastfeeding and not breastfeeding.
The 2.3-kg-greater weight loss of the intervention group compared with standard care may seem modest but should be considered in the context of a young adult, postpartum population. Weight gains of 0.8 kg to 1 kg per year in young adults has been found to increase cardiovascular disease risk, and 2% to 5% weight loss results in improved systolic blood pressure, glucose, and triglycerides. Even modest (≥1 kg) postpartum weight retention has been linked to increased risk of later weight gain and development of obesity and diabetes in women, and the intervention group increased by 14.2 percentage points the proportion of women achieving preconception weight or below over 12 months. Future research is needed to assess the effect of this intervention’s postpartum weight changes on long-term health.
There were no significant group differences in dietary intake or physical activity, which is surprising given the effects on weight loss in the intervention group. Self-reported dietary intake is consistently underreported among women with obesity and dieters. The ASA24 dietary recall instrument has been validated but not in primarily Hispanic populations. It is possible that dietary intake was reduced among intervention participants but not captured by the measurement instrument. Physical activity was objectively measured using a waist-worn accelerometer. Although this method has some limitations, accelerometers provide reliable estimates of physical activity. The reasons behind the lack of an intervention effect on activity remain unclear; the intervention targeted the most commonly reported barriers in postpartum women, including time, child care, and fatigue. The intervention provided pedometers, encouraged “baby-friendly” and inexpensive lifestyle activities, and gradually increased in physical activity. Other research has highlighted the potential importance of health professionals and partner support in promoting postpartum physical activity. These elements and other dietary instruments could be studied in future interventions.
A potential benefit of a primarily internet-based intervention is the relatively low delivery cost compared with traditional programs requiring more frequent face-to-face contact. As an exploratory analysis, the payer’s cost was estimated at $276 per participant (about $125 per kilogram lost) and may be less expensive than traditional face-to-face weight loss interventions. However, future cost-effectiveness trials are needed.
This study had several strengths. It is a long-term, adequately powered study of an internet-based weight control program for low-income postpartum women delivered in WIC. The sample included predominantly Hispanic women who face food insecurity. The intervention was developed in consultation with the WIC program. The study used a standard care control group, which has been lacking in other studies. Also, measures were collected by assessors who were masked to randomization, and retention was high at all time points (≥89%).
Limitations
This study had several limitations. The study sample was restricted to women in the WIC program, and some could not be reached (25%) or refused participation (17%). The study provided some participants (n = 63; 36.2%) with internet access, which could be cost-prohibitive; however, internet access has increased steadily since 2010 to more than 74% of low-income households and 81% of the Hispanic population. The study groups were not matched on contact, so it remains unknown if the weight differences were attributable to actual intervention components. The study design tested a treatment “package” and did not allow for isolation of the independent contribution of discrete intervention components. The intervention did not include an app, which might have enhanced utilization. To gain efficiency, linear mixed-effects models used all available data, including from participants with missing data. Although attrition was minimal (10.8%) and sensitivity analyses were consistent, it is possible that the results could be biased if the data lost to follow-up were not missing at random. The sensitivity analysis resulted in a mean 1.6-kg loss and P value of .03; these fell short of the loss of 2 kg or more considered clinically important and the Bonferroni-corrected threshold of P = .005. In addition, as women likely underreported preconception weight, the intervention’s effect on return to prepregnancy weight might also be inaccurate.
Conclusions
Among low-income postpartum women, an internet-based weight loss program in addition to the Special Supplemental Nutrition Program for Women, Infants, and Children (WIC) compared with the WIC program alone resulted in a statistically significant greater weight loss over 12 months. Further research is needed to determine program and cost-effectiveness as part of the WIC program.
References
- 1.Rasmussen KM, Yaktine AL; Institute of Medicine (US) . Weight Gain During Pregnancy: Reexamining the Guidelines. Washington, DC: National Academies Press; 2009. [PubMed] [Google Scholar]
- 2.Walker L, Freeland-Graves JH, Milani T, et al. . Weight and behavioral and psychosocial factors among ethnically diverse, low-income women after childbirth: II: trends and correlates. Women Health. 2004;40(2):19-34. [DOI] [PubMed] [Google Scholar]
- 3.Olson CM, Strawderman MS. The relationship between food insecurity and obesity in rural childbearing women. J Rural Health. 2008;24(1):60-66. [DOI] [PubMed] [Google Scholar]
- 4.Gunderson EP, Murtaugh MA, Lewis CE, Quesenberry CP, West DS, Sidney S. Excess gains in weight and waist circumference associated with childbearing: the Coronary Artery Risk Development in Young Adults Study (CARDIA). Int J Obes Relat Metab Disord. 2004;28(4):525-535. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Rooney BL, Schauberger CW, Mathiason MA. Impact of perinatal weight change on long-term obesity and obesity-related illnesses. Obstet Gynecol. 2005;106(6):1349-1356. [DOI] [PubMed] [Google Scholar]
- 6.Villamor E, Cnattingius S. Interpregnancy weight change and risk of adverse pregnancy outcomes: a population-based study. Lancet. 2006;368(9542):1164-1170. [DOI] [PubMed] [Google Scholar]
- 7.National Institute for Health and Care Excellence NICE Public Health Guidance: Dietary Interventions and Physical Activity Interventions for Weight Management Before, During and After Pregnancy. London, England: NICE; 2010. [Google Scholar]
- 8.Krummel D, Semmens E, MacBride AM, Fisher B. Lessons learned from the mothers’ overweight management study in 4 West Virginia WIC offices. J Nutr Educ Behav. 2010;42(3)(suppl):S52-S58. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Walker LO, Sterling BS, Latimer L, Kim SH, Garcia AA, Fowles ER. Ethnic-specific weight-loss interventions for low-income postpartum women: findings and lessons. West J Nurs Res. 2012;34(5):654-676. [DOI] [PubMed] [Google Scholar]
- 10.Chang MW, Nitzke S, Brown R. Design and outcomes of a Mothers In Motion behavioral intervention pilot study. J Nutr Educ Behav. 2010;42(3)(suppl):S11-S21. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Østbye T, Krause KM, Lovelady CA, et al. . Active mothers postpartum: a randomized controlled weight-loss intervention trial. Am J Prev Med. 2009;37(3):173-180. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Herring SJ, Cruice JF, Bennett GG, Davey A, Foster GD. Using technology to promote postpartum weight loss in urban, low-income mothers: a pilot randomized controlled trial. J Nutr Educ Behav. 2014;46(6):610-615. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Manzoni GM, Pagnini F, Corti S, Molinari E, Castelnuovo G. Internet-based behavioral interventions for obesity: an updated systematic review. Clin Pract Epidemiol Ment Health. 2011;7:19-28. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Bennett GG, Steinberg DM, Stoute C, et al. . Electronic health (eHealth) interventions for weight management among racial/ethnic minority adults: a systematic review. Obes Rev. 2014;15(suppl 4):146-158. [DOI] [PubMed] [Google Scholar]
- 15.Phelan S, Brannen A, Erickson K, et al. . “Fit Moms/Mamás Activas” internet-based weight control program with group support to reduce postpartum weight retention in low-income women: study protocol for a randomized controlled trial. Trials. 2015;16:59. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Tate DF, Jackvony EH, Wing RR. A randomized trial comparing human email counseling, computer-automated tailored counseling, and no counseling in an internet weight loss program. Arch Intern Med. 2006;166(15):1620-1625. [DOI] [PubMed] [Google Scholar]
- 17.Tate DF, Jackvony EH, Wing RR. Effects of internet behavioral counseling on weight loss in adults at risk for type 2 diabetes: a randomized trial. JAMA. 2003;289(14):1833-1836. [DOI] [PubMed] [Google Scholar]
- 18.US Department of Agriculture WIC: The Special Supplemental Nutrition Program for Women, Infants, and Children. Washington, DC: Food and Nutrition Service; 2008. [Google Scholar]
- 19.Diabetes Prevention Program (DPP) Research Group The Diabetes Prevention Program (DPP): description of lifestyle intervention. Diabetes Care. 2002;25(12):2165-2171. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Ryan DH, Espeland MA, Foster GD, et al. ; Look AHEAD Research Group . Look AHEAD (Action for Health in Diabetes): design and methods for a clinical trial of weight loss for the prevention of cardiovascular disease in type 2 diabetes. Control Clin Trials. 2003;24(5):610-628. [DOI] [PubMed] [Google Scholar]
- 21.Stevens-Simon C, McAnarney ER, Coulter MP. How accurately do pregnant adolescents estimate their weight prior to pregnancy? J Adolesc Health Care. 1986;7(4):250-254. [DOI] [PubMed] [Google Scholar]
- 22.Phelan S, Phipps MG, Abrams B, Darroch FE, Wing RR. Randomized trial of a behavioral intervention to prevent excessive gestational weight gain: the Fit for Delivery Study. Am J Clin Nutr. 2011;93(4):772-779. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Kirkpatrick SI, Subar AF, Douglass D, et al. . Performance of the automated self-administered 24-hour recall relative to a measure of true intakes and to an interviewer-administered 24-h recall. Am J Clin Nutr. 2014;100(1):233-240. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.National Cancer Institute Automated Self-Administered 24-hour Hour dietary assessment tool. https://riskfactor.cancer.gov/tools/instruments/asa24.html. Accessed May 30, 2017.
- 25.Sasaki JE, John D, Freedson PS. Validation and comparison of ActiGraph activity monitors. J Sci Med Sport. 2011;14(5):411-416. [DOI] [PubMed] [Google Scholar]
- 26.Choi L, Liu Z, Matthews CE, Buchowski MS. Validation of accelerometer wear and nonwear time classification algorithm. Med Sci Sports Exerc. 2011;43(2):357-364. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Bureau of Labor Statistics Occupational Employment and Wages, May 2016: 21-1094 Community Health Workers. 2016. https://www.bls.gov/oes/current/oes211094.htm. Accessed April 28, 2017.
- 28.Bureau of Labor Statistics Occupational Employment and Wages, May 2016: 15-1131 Computer Programmers. 2017. https://www.bls.gov/oes/current/oes151131.htm. Accessed April 28, 2017.
- 29.Truesdale KP, Stevens J, Lewis CE, Schreiner PJ, Loria CM, Cai J. Changes in risk factors for cardiovascular disease by baseline weight status in young adults who maintain or gain weight over 15 years: the CARDIA study. Int J Obes (Lond). 2006;30(9):1397-1407. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Wing RR, Lang W, Wadden TA, et al. ; Look AHEAD Research Group . Benefits of modest weight loss in improving cardiovascular risk factors in overweight and obese individuals with type 2 diabetes. Diabetes Care. 2011;34(7):1481-1486. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Phelan S, Smith K, Steele J-M, Wilt D, Ames S, McClure L. What type of weight loss program do postpartum women want? treatment preferences of postpartum women in two community settings. Calif J Health Promot. 2010;8(1):1-11. https://pdfs.semanticscholar.org/2ebc/c90fcd00aa830266ef1c872780936db533fc.pdf. Accessed May 26, 2017.25750596 [Google Scholar]
- 32.Smeeth L, Ng ES. Intraclass correlation coefficients for cluster randomized trials in primary care: data from the MRC Trial of the Assessment and Management of Older People in the Community. Control Clin Trials. 2002;23(4):409-421. [DOI] [PubMed] [Google Scholar]
- 33.Feise RJ. Do multiple outcome measures require P-value adjustment? BMC Med Res Methodol. 2002;2:8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Perneger TV. What’s wrong with Bonferroni adjustments. BMJ. 1998;316(7139):1236-1238. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Hill RJ, Davies PSW. The validity of self-reported energy intake as determined using the doubly labelled water technique. Br J Nutr. 2001;85(4):415-430. [DOI] [PubMed] [Google Scholar]
- 36.Trabulsi J, Schoeller DA. Evaluation of dietary assessment instruments against doubly labeled water, a biomarker of habitual energy intake. Am J Physiol Endocrinol Metab. 2001;281(5):E891-E899. [DOI] [PubMed] [Google Scholar]
- 37.Plasqui G, Bonomi AG, Westerterp KR. Daily physical activity assessment with accelerometers: new insights and validation studies. Obes Rev. 2013;14(6):451-462. [DOI] [PubMed] [Google Scholar]
- 38.Evenson KR, Aytur SA, Borodulin K. Physical activity beliefs, barriers, and enablers among postpartum women. J Womens Health (Larchmt). 2009;18(12):1925-1934. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Griffiths F, Lindenmeyer A, Powell J, Lowe P, Thorogood M. Why are health care interventions delivered over the internet? a systematic review of the published literature. J Med Internet Res. 2006;8(2):e10. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Jakicic JM, Tate DF, Lang W, et al. . Effect of a stepped-care intervention approach on weight loss in adults: a randomized clinical trial. JAMA. 2012;307(24):2617-2626. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Pew Research Center Americans' internet access: 2000-2015. http://www.pewinternet.org/2015/06/26/americans-internet-access-2000-2015/. Accessed May 1, 2017.
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.