Are evidence-based, community-engaged energy balance interventions enough for extremely vulnerable populations?

Abstract

Well-documented associations between lifestyle behaviors and disease outcomes necessitate evidence-based health promotion interventions. To enhance potential efficacy and effectiveness, interventionists increasingly respond to community priorities, employ comprehensive theoretical frameworks, invest heavily to ensure cultural fit, implement evidence-based programming, and deploy research gold standards. We describe a project that followed all of these recommended strategies, but did not achieve desired outcomes. This community-based participatory research (CBPR) energy balance (diet and physical activity) intervention, conducted in Appalachian Kentucky among 900+ residents, employed a wait list control cluster randomized design. We engaged faith institutions, took an intergenerational approach, and modified two existing evidence-based interventions to enhance cultural relevance. Despite these efforts, fruit and vegetable consumption and physical activity did not change from baseline to post-test or differed significantly between intervention and wait list control groups. Barriers to engaging in optimal energy balance focused more on motivation and attitude than on structural and material barriers. The complex interplay of psychosocial, structural, and physiological processes offers significant challenges to groups with entrenched health challenges.

Despite following rigorous intervention standards, our diet and physical activity intervention conducted among 900+ rural Appalachian residents did not change behavior. We explore possible reasons for this lack of change.

Implications

Practice: Practitioners may want to focus less on educational and logistical barriers to behavior change and instead develop programs to inculcate positive norms and attitudes into the community; examples include sponsoring a walking group program, developing friendly competitions to increase healthy eating, and working with community partners—faith groups, community centers and schools, to employ innovative culture change programs and policies.

Policy: Our finding—that motivational and attitudinal barriers persisted and undermined energy balance—implies that policymakers can address attitudinal factors through social marketing approaches, public service and educational campaigns, and legislation designed to make energy balance the optimal default.

Research: Our finding—that despite following gold standards, this intervention did not significantly change energy balance behaviors or outcomes—requires researchers to reconsider core assumptions, including the promise of evidence-based interventions and the accuracy of physical activity and dietary intake instruments.

INTRODUCTION

The association between optimal lifestyle and disease prevention has been well-established. Less is known about how to develop and implement interventions that improve these complex behaviors [1]. Community-based participatory research (CBPR), cultural adaptation, and theoretically-based research designs have been recommended to foster improved energy balance [2, 3]. Cultural adaptations tailor intervention messages to respond to cultural metaphors and concepts [4, 5]. Adapted interventions account for the culturally variable patterns that shape health behavior decisions [6]. We sought to determine whether a community-engaged, culturally-adapted intervention implemented with rigorous training and fidelity would promote diet and physical activity (PA) (energy balance) behavior change. The study was located in Appalachian Kentucky, which maintains extremely low socioeconomic status and health indicators. Poverty rates in the six study counties are approximately twice that of the nation (25.2% vs. 15.4%). The per capita annual income is significantly lower than national average ($30,308 vs. $46,049), as are rates of employment and educational attainment [7]. Nearly half (46.2%) and one-quarter (24.9%) of Kentucky adults consume fewer than one daily serving of fruits and vegetables (f/v), respectively. Over half (54.0%) do not obtain at least 150 min of weekly moderate intensity PA. Over one-third (35.1%) of Kentucky adults are overweight, and 31.6% are obese, even higher in Appalachian Kentucky [8].

METHODS

A community-engaged, culturally adapted intervention

The intervention was based on programmatic components previously successful in our research, including staffing by local interviewers and Lay Health Advisors (LHAs); emphasizing traditional Appalachian strengths of social support, faith, and family; and including scriptural references, storytelling; and witnessing [9]. As summarized in Table 1, the intervention addressed individual, interpersonal, community, and organizational levels of the Socioecological Model (SEM) and Social Cognitive Theory (SCT), though public policy was not emphasized.

Table 1.

Incorporation of theoretical frameworks (SCT and SEM) into intervention

Theoretical component	Intervention component
SCT
Situation—Individual’s perception of the environment	Intrapersonal: Correct misperceptions, promote healthful norms through LHA workshops
Behavioral capability—Ability to carry out new behavior	Intrapersonal: Promote mastery learning through skills training through LHA workshops and tailored MI counseling; decisional balance negotiated with LHAs.
Expectations—outcomes anticipated	Interpersonal: Model positive outcomes through participant discussion of barriers to healthy energy balance and group problem-solving.
Value Expectancies—internal value placed on behavioral performance	Intrapersonal: Present outcomes in discussions/workshops.
Self-control—personal regulation of goal-directed behavior	Intrapersonal: Self-monitoring, goal setting, problem solving, self-reward: church workshops and tailored MI. Faith-based group reinforce behavior and messages.
Observational learning—watching the actions and outcomes of others’ behavior	Interpersonal/social network: Group discussions and witnessing
Reinforcements—Responses to behavior that influence the likelihood of recurrence	Intrapersonal and interpersonal: LHA: church workshops, tailored MI, witnessing
Self-efficacy—The person’s confidence in performing a particular behavior	Intrapersonal: LHA church workshops and tailored MI. Storytelling.
Emotional coping responses—Strategies to deal with emotional stimuli	Intrapersonal: LHA workshops and tailored MI counseling, Witnessing, storytelling
Reciprocal determinism—Dynamic interaction of person, behavior, environment where behavior is performed.	Environmental, skills, and personal change: LHA church workshops and tailored MI.
SEM
Intrapersonal	See above
Interpersonal	See above
Community	Development of community cookbook; distribution of community resource guide.
Organization	Location in churches and community centers

MI motivational interviewing; LHA lay health advisor; SCT social cognitive theory; SEM socioecological model.

We conducted focus groups, in-depth interviews, and community forums to culturally adapt We Can! (“Ways to Enhance Children’s Activity & Nutrition”) and Media Smart Youth (https://www.nichd.nih.gov/msy/Pages/index.aspx). The adapted We Can! program was designed for parents and caregivers, while Media Smart Youth was targeted at children. Both evidence-based programs [10] consisted of 6 weeks of hands-on energy balance workshops. Workshops focused on key concepts such as controlling portion size, reading food labels, learning new physical activities, and “weaning the screen.” Two LHAs led the workshops, which included 12–18 participants and was delivered in a “learning, activity, food, fun” format. The LHAs returned to their assigned groups each week ensuring consistency and instilling trust. Other cultural elements included storytelling that “witnessed” about good energy balance, newsletters, and a community cookbook.

Recruitment and randomization

Churches were selected from a comprehensive county-wide list of 272 religious organizations. Consistent with the Appalachian religious environment, the list included exclusively Christian congregations. Five additional community organizations were included for those individuals unaffiliated with a church. Churches and other organizations provided a location for recruitment and intervention conduct. To recruit the churches, project staff met with the minister or representative, explained the project, and answered questions. Interested ministers were asked to provide the name of a liaison who could partner with our project. Liaisons strategized with the project managers to strengthen program implementation.

Most sites held project sign-up events where our staff determined eligibility. Eligibility included being age 8+, an Appalachian resident, an English speaker, and intending on living in region throughout the study. Staff arranged follow-up informed consent and baseline interviews. Entire households were encouraged to participate if they met the eligibility requirements. Here, we report on outcomes for adults only.

Forty-three churches or community organizations, approximately 16% of the possible churches and community centers in the area, were enrolled in the intervention. This sample size allowed us to achieve 80% power to detect the difference of 10% in proportions of participants engaging recommended levels of PA and fruit and vegetable (f/v) intake. Blocked randomization of individuals with variable block sizes ensured equal probability of group assignment. Our biostatistician conducted the computer-generated randomization.

Study design

We conducted a single-blinded, two-arm cluster randomized community trial with a wait-list control design. Treatment and wait-list control participant groups completed a baseline assessment. Afterwards, the treatment group participants received the intervention and the wait-listed participants received an educational luncheon focused on stress reduction. Participants from both groups completed post-test 1. Wait-listed participants then received the intervention (beginning approximately 14 weeks after baseline assessment). Six months following the completion of their intervention activities, all participants completed the post-test 2 (exit) assessment.

Training, fidelity, and data transfer

LHAs, demographically similar to most adult participants (i.e., female, middle-aged, married, middle-to-lower socioeconomic status) underwent a rigorous 4-day training on energy balance, protection of human subjects, intervention protocols, and Motivational Interviewing. At the beginning, two project managers attended intervention workshops every other week and employed a fidelity and content checklist. If there was meaningful divergence from the curriculum, project managers retrained LHAs in relevant content. LHAs unable to improve after three retrainings were dropped from the program. LHAs met monthly to strategize about challenges and attended a full day refresher course 4 months into the program. Fidelity and quality was assessed further by periodic debriefing interviews with participants.

Pre- and post-test data collection was undertaken by experienced interviewers who were trained on project goals, human subjects protection, data collection, and data transfer. Interviewers collected data in participants’ homes, the project office, or churches. Due to literacy concerns, questionnaires were read aloud. Throughout, feedback was provided to the interviewers to improve their performance. Project leaders held monthly meetings with the interviewers as a group to review the protocol and provide solutions to common problems. Staff (not interviewers) merged the databases, imported them into SPSS, and cleaned the data using a syntax program.

Measures

All study instruments have been validated in diverse populations. We pilot tested all instruments with our Community Advisory Board members and local staff. Baseline and post-tests 1 and 2 assessments consisted of closed-ended questions on sociodemographics, health history, health status, basic knowledge, self-efficacy, decisional balance, barriers, and f/v intake and PA behavior [11]. For each completed baseline questionnaire, the participants received $10, a standard rate in the region. The two primary outcomes were engagement in recommended levels (i) of leisure-time aerobic PA and (ii) f/v intake.

Data analyses

We used descriptive statistics to characterize the study sample, including PA levels, f/v intake, and barriers to PA and f/v consumption. Because the study was designed as a cluster randomized trial, we used cluster-adjusted chi-square and t-tests to assess baseline group differences. To test the effect of intervention on the main outcomes, we used three sets of binary logistic regression models. The first set included a dummy variable for group assignment (treatment vs. wait-list control) and baseline levels of participant’s PA engagement (or f/v intake) as fixed effects only. In the second set of models, a random intercept for church was added to account for participants’ clustering in churches. The third set of models was expanded to account for participants’ personal and health characteristics selected post hoc.

We also conducted secondary analyses using random-intercept models to test differences in mediators of behavior change following the intervention (i.e., at post-test 1). The mediators included PA self-efficacy score, f/v consumption self-efficacy score, and barriers to engaging in PA and consuming f/v. These models were adjusted for participant’s baseline score as a fixed effect covariate and church assignment as a random effect. We followed intent-to-treat principles. We retained participants lost to follow up in the analyses with the assumption that they did not increase their PA and f/v intake to recommended levels. Analyses were conducted using Stata/SE 14.1 (StataCorp, Inc.) including the packages for performing cluster-adjusted chi-square (chchi2) and t-tests (clttest). Significance level was set α of 0.05, and all tests were two-tailed.

RESULTS

Our sample consisted of 905 participants from 43 churches or community organizations. Twenty-one churches/organizations with 468 participants were cluster-randomized to treatment and 22 churches/organizations with 437 participants cluster-randomized to wait-list control groups. Of those participants, 25 (2.8%) were lost to follow up, including 20 from the treatment and 5 from the wait-list control group. No statistically significant differences existed between the groups except that higher percentage of wait-list control participants were employed compared with the treatment group (45.1% vs. 30.6%, p = .03).

Primary outcomes

Although not statistically significant, fewer wait-list control group participants (5.1%) than treatment group participants (7.2%) reported engaging in recommended PA levels at baseline (Table 2). In the treatment group, 8.1% and 8.4% of participants reported meeting national recommendations for PA at post-tests 1 and 2, respectively. In the wait-list control group, 7.2% and 5.5% met national PA recommendations at post-tests 1 and 2, respectively. At baseline, 33.5% in the treatment and 27.0% in the wait-list group adhered to the national recommendation for f/v intake (p > .05). Nearly half (44.4% and 42.6%) of the treatment group participants reported meeting recommendations on weekly f/v intake at post-tests 1 and 2, respectively. Among the wait-list participants, 33.7% reported meeting national recommendations at post-test 1 and 43.2% at post-test 2.

Table 2 .

Primary outcomes: physical activity levels by intervention assignment

	Groups
	Treatment	Wait-list control
	n (%a)	n (%a)
Sufficiently active/meet national recommendations
Baseline (n = 460, 435)	33 (7.2) ns	22 (5.1) ns
Follow-up 1 (n = 446, 431)	36 (8.1)	31 (7.2)
Exit (n = 441, 422)	37 (8.4)	23 (5.5)
Respondents meeting recommendations on weekly fruits and vegetables intake
Baseline (n = 451, 423)	151 (33.5) ns	114 (27.0) ns
Follow-up 1 (n = 430, 422)	191 (44.4)	142 (33.7)
Follow-up 2 (n = 434, 414)	185 (42.6)	179 (43.2)

ns no statistically significant difference in percentage of participants by assignment at baseline, cluster-adjusted p-value > .05.

^aDenominator is a total number of participants in each intervention group at each assessment point.

As indicated by the estimates from the logistic regressions adjusted for the church organization random effect (Table 3), there were no statistically significant intervention effects on PA levels and f/v intake at post-test 1 (i.e., after the treatment group participants received the intervention and the wait-listed participants had not).

Table 3.

Odds ratios of engaging in sufficient/recommended levels of physical activity and meeting recommended levels of fruits and vegetables intake for treatment group versus wait-list control group at post-test 1

Models	PA levels		Recommended f/v intake
	OR (95%)	p-value	OR (95%)	p-value
Model 1a: Adjusted for baseline levels only as fixed effects	0.98 (0.58–1.67)	.95	1.45 (1.07–1.97)	.02*
Model 2b: Adjusted for baseline levels as fixed effects and church (random effect/ intercept)	0.97 (0.56–1.69)	.91	1.59 (0.74–3.42)	.23
Model 3c: Adjusted for baseline levels and participants characteristics (fixed effects) and for church (random effect/intercept)	0.81 (0.42–1.59)	.55	1.49 (0.65–3.39)	.35

CI confidence interval; OR odds ratio; PA physical activity.

^*Statistically significant difference; significance level was set at α = 0.05.

^aModel 1 represents the primary outcome analysis, providing the adjusted OR for treatment effect accounting for participants baseline levels (n’s = 858; 814).

^bModel 2 represents the primary outcome analysis, providing the adjusted OR for treatment effect accounting for baseline levels and participants’ clustering in churches (n’s = 851; 807).

^cModel 3 provides the adjusted OR for treatment effect accounting for participants’ baseline levels, clustering in churches, and additional participant characteristics selected post hoc (n’s = 793; 766).

Secondary analyses

At baseline, there were no statistically significant differences in mean scores of behavior change mediators between treatment and wait-list control group participants. Based on the results of random intercept-only models, there were no statistically significant differences in mediators of behavior and psychosocial change between treatment versus wait-list control groups at post-test 1 (results not shown).

Barriers to behavioral change

Participants were more likely to indicate that internal or “attitudinal” factors were responsible for lack of behavioral change. Such factors include lack of motivation, lack of energy, not wanting to change habits, and feeling better when eating unhealthy food. The least frequently reported barriers to behavior change involved structural and logistical issues.

Discussion

Despite our efforts to address community health priorities, deploy evidence-based interventions grounded in behavioral theory and research gold standards, we observed few statistically significant changes in increasing PA or f/v intake. We offer four possible explanations for these null findings. First, insufficient evidence and measurement challenges complicate a definitive conclusion. Second, our greater emphasis on the SEM over the SCT may have been problematic. Third, even evidence-based and culturally adapted complex interventions may be insufficient for populations bearing a heavy burden of health disadvantage. Finally, we may have underestimated participants’ tremendous disease burden and overall physiologic challenges.

Insufficient evidence of lack of intervention effect and continued measurement challenges

We may have been unable to accurately measure the effects of the intervention due to insufficient data against the null hypothesis. Measurement issues pose challenges. For example, participants may have overestimated their PA levels during the baseline interview, classifying common household chores or a stroll to the mailbox as PA [12]. Over the course of the intervention, participants may have learned more standard classification of PA, so that by the final assessment, they reported their PA levels more consistently with these standards. Future intervention testing should incorporate more detailed explanations of PA during the baseline data collection in order to more accurately measure intervention effects.

A problematic emphasis on the SEM

The well-documented challenging conditions of Appalachian life led us to emphasize components of the SEM over the SCT, leading to a family and organizationally-based intervention. Although participants were satisfied with the program and had low rates of attrition, this inward focus may have unwittingly undermined its success. Recently, researchers have suggested that strong family and community ties may offer both “support and sabotage” [13] through influential social norms, resources, and behavioral patterns. Future energy balance interventions could adopt social support strategies from peer provided mental health services—to employ the shared experiences of a condition, but using the changed health behaviors of a peer to promote “recovery” [14].

Too little, too late?

Our results suggest that populations bearing a heavy burden of health disadvantage may require enhancing intervention dose. Research has demonstrated that enhancing the dose of the intervention may produce better outcomes. At the same time, convincing evidence exists that shorter programming may also positively affect lifestyle [15, 16].

Our results also converge with a growing body of research on the physiological processes of inflammation that call in question previous formulations of energy balance. Increased attention must be paid to the metabolic, inflammatory, and hormonal pathways shaping obesity that interact with structural causes of poor energy balance [17]. Changing diet may be especially challenging for people who are already overweight or obese. As obese people try to reduce caloric intake—oftentimes through increasing f/v consumption and PA—their hormonal pathways change, causing them to face increased appetite and desire to eat [18]. Being overweight itself may constitute a barrier to PA [19]. BMI may actually predict sedentary behavior [20]. Research drawing attention to the socioeconomic factors shaping the biobehavioral processes of energy balance holds further relevance for vulnerable populations—urging attention to the relationships between inequality and physiology. Enduring insecurities in basic needs result in emotional and stress responses in endocrine, immune, and nervous systems. Researchers have established the physiologic mechanism linking stress and uncontrolled eating and reduced cognitive resources to choose healthy foods [21]. Indeed, most barriers endorsed by our participants emphasized internal or cognitive factors including lack of motivation, lack of energy, not wanting to change habits, and feeling better when eating unhealthy food. Our intervention incorporated many evidence-based interpersonal and intrapersonal components that should have addressed these barriers. However, our study suggests that self-efficacy and physiological challenges may limit the effectiveness of these strategies. Barriers related to structural issues—the least reported barriers to behavior change in our study—have been the focus of rural health disparities research but such barriers appear to be only one of a series of challenges to behavioral change.

Future investigations would benefit from a more inclusive sample and more robust incorporation of levels influential in behavior. While our predominantly white sample was appropriate for studying Appalachians, we are unable generalize results to other groups. Finally, we acknowledge the limitation of a predominantly female sample.

Future interventions must consider incorporating multiple levels of influences to address these deeply embedded barriers that cannot be addressed by cognitive, social, and ecological strategies alone. “Moving the needle” on such challenging behaviors in underserved populations may require tailored and even individualized approaches to dietary change and weight loss to address physiological factors shaping energy balance behaviors alongside structural change to promote improved PA and f/v consumption.

Compliance with Ethical Standards

Conflict of Interest: The authors declare that they have no conflict of interest.

Primary data: These findings have not been previously published and the manuscript is not under consideration for publication elsewhere. There has been no previous reporting of these data. The authors have full control of all primary data and agree to let TBM review our data if requested.

Ethical Approval: Human subjects protection protocols, including implementation of informed consent documents and extensive training, were overseen by the University of Kentucky Office of Research Integrity, Protocol #08-0119-P2H. The research was conducted according to the ethical standards of the 1964 Declaration of Helsinki and its later amendments or comparable ethical standards.

Informed Consent: Informed consent was obtained from all individual participants included in the study. This article does not contain any studies with animals performed by any of the authors.