The Faith, Activity and Nutrition (FAN) Dissemination and Implementation Study: Phase 1 Implementation Monitoring Methods and Results

Abstract

Faith-based settings offer opportunities for reaching populations at risk for chronic conditions and are optimal settings for dissemination and implementation (D&I) research. Faith, Activity, and Nutrition (FAN) is an evidence-based program designed to promote physical activity (PA) and healthy eating (HE) through church policy, systems, and environmental change. We report implementation fidelity for Phase 1 of the FAN D&I project, a countywide effort. The group randomized study included pre- and post-intervention assessments of core PA and HE components. We compared implementation in early intervention (n=35) versus delayed intervention (control, n=19) churches; assessed individual church implementation; and examined the effects of level of implementation on church member outcomes. Implementation assessments were conducted with the FAN coordinator via telephone survey. Study outcomes were assessed with church members 8–12 months following baseline assessment via self-administered surveys. We found significantly higher levels of implementation for PA opportunities, PA and HE guidelines, PA and HE messages, and PA and HE pastor support in intervention versus control churches and showed church-level variation in PA and HE implementation. PA self-efficacy varied by level of implementation; high and low implementers did not differ in proportion of church members physically inactive, although low implementers had fewer members inactive than controls. The high level of implementation in intervention churches shows promise for broader dissemination of FAN.

Introduction

Faith-based settings have great potential for reaching populations at risk for multiple chronic conditions that can be ameliorated through increased physical activity (PA) and healthy eating (HE) (Gigler, Appel, Davidhizar, & Davis, 2008; Kegler et al., 2013; Levin, 2013). Systematic reviews indicate promise for promoting PA (Bopp, Peterson, & Webb, 2013; Parra, Porfirio, Arrendondo, & Attallah, 2017) and addressing obesity (Lancaster, Carter-Edwards, Grilo, Shen, & Schoenthaler, 2014; Maynard, 2017) in faith settings but note the need for increased research rigor. Faith-based settings also afford opportunities for sustainable change through intervention approaches that address organizational and systems change. For example, intervention approaches in faith-based settings can include pastors, increase capacity of church-level change agents, deliver interventions through lay health leaders, and target specific organizational and environmental factors that influence PA and HE (Campbell et al., 2007). This approach is consistent with community interventions that target policy, systems, and environmental change which can reach more people by focusing on social and environmental forces that facilitate and impede healthy behaviors (Bunnell et al., 2012).

Dissemination research targets the distribution of evidence-based interventions to specific public health settings, whereas implementation research investigates processes and factors associated with successful integration of evidence-based policies and practices in real-world public health settings (Rabin, Brownson, Haire-Joshu, Kreuter, & Weaver, 2008). Implementation research includes an assessment of the extent to which core components of the intervention were implemented (i.e., implementation fidelity) in the real-world setting (Rabin et al., 2008). It is important to monitor implementation processes in methodologically sound ways, including using a strong study design (i.e., pre- and post-measures in randomly assigned control and intervention groups), assessing implementation fidelity (Yeary, Klos, & Linnan, 2012), and using criteria for evidence of implementation set a priori (Saunders, 2016). However, it is rare to find implementation studies that have all these design elements.

Faith, Activity, and Nutrition (FAN) is an evidence-based program which implements policy, systems, and environmental change in church-based settings to promote PA and HE among church congregants and has previously been shown to have positive impacts on PA and fruit and vegetable (FV) intake in churches in South Carolina (Wilcox et al., 2010; Wilcox et al., 2013). The FAN Dissemination and Implementation (D&I) study is examining D&I processes for FAN in two phases. FAN D&I Phase 1 (completed) involved churches in a single county, whereas Phase 2 (ongoing) is a statewide initiative (Wilcox et al., 2018).

The FAN D&I Phase 1 intervention was effective, with intervention church members reporting greater post-intervention church-level PA opportunities, PA and HE messages, and PA and HE pastor support, but not FV opportunities, which were relatively high in both groups at baseline and post-intervention. Post-intervention, the proportion of physically inactive members was lower in intervention versus control churches. FV self-efficacy, FV intake, and the proportion meeting PA guidelines did not differ by group (Wilcox et al., 2018).

The study reported here focuses on Phase I implementation processes. Specifically, the purposes of this paper are to describe the implementation fidelity monitoring methods and results in the FAN D&I Phase 1 study. Future papers will address factors influencing implementation processes.

Method

FAN D&I Intervention and Implementation Approach

With guidance from Cohen’s structural model of health behavior (Cohen, Scribner, & Farley, 2000), FAN D&I worked with churches to identify strategies to promote PA and HE with an emphasis on: 1) availability and accessibility of products associated with protective health outcomes, including providing PA opportunities before, during, or after worship services and/or church events and healthy food options at church events; 2) characteristics of products ensuring that opportunities are appealing, convenient, and appropriate for the faith setting; 3) social structures and policies to promote healthy behaviors through organizational guidelines and support (e.g., guidelines for serving FVs at church meals; including an active break in all church meetings longer than 30 minutes); and 4) media and cultural messages including bulletin inserts, messages during worship services, and bulletin boards that promote PA and HE (Wilcox et al., 2007; Wilcox et al., 2010; Wilcox et al., 2013; Wilcox et al., 2018) (see Table 1).

Table 1.

FAN Intervention Conceptual Model and Core Elements

Cohen’s Structural Model Domain	Physical Activity Core Elements	Healthy Eating Core Elements
Protective products or services	PA Opportunities Provide opportunities for PA before, during, or after church services and/or events Consider: Offer PA program(s) that will reach most members of church	HE Opportunities Serve more fruits, vegetables, and whole grain foods and serve lower-fat and lower sodium foods at church meals and snacks
Characteristics of products or services	Choose Relevant and Enjoyable Activities Consider ways to make sure that the chosen PA/HE activities are appealing to members (examples listed which include tie health messages to scripture, make opportunities convenient & appropriate by surveying members, involve youth, use contests, ensure foods are flavorful, sample healthy foods, etc.)
Social structures and policies	Suggest guidelines and practices that the Pastor can put in place at church to support physical activity and healthy eating. Here are some examples of church practices: Healthy food options must be available at every church function that has meals or snacks Physical activity breaks must be built into every meeting lasting 30 minutes or longer Church menus for all church events must be approved by the Church Cook, Menu planner, or FAN Committee Church social events must include active games or opportunities for youth (although adults are also encouraged to take part!) Include water and low calorie drinks as the primary beverage options at church events
Media and cultural messages	Get the Message Out (Media) The following activities are required as part of the FAN program (additional activities can be chosen with examples provided): Pass out bulletin inserts (or include information in bulletins) at least monthly about physical activity and healthy eating to the whole church (provided by the FAN program) Share messages about physical activity and healthy eating during church service Create a bulletin board in a highly trafficked area that includes information about physical activity and healthy eating as well as opportunities to be active and eat well Consider choosing one more activities (e.g., pass out handouts, make church announcements about PA/HE, have FAN coordinator attend church meetings to talk about PA/HE, dispel myths, etc)
	Engage and Support Pastor (Cultural Messages) The following activities are strongly recommended as part of the FAN program (additional activities can be chosen with examples provided): Share the monthly Pastor activity (located in church resource binder) with Pastor and encourage him/her to try it and share his/her learnings with the congregation Ask the Pastor to allow the FAN Coordinator (or Champions) to have time to talk about physical activity and healthy eating during worship services and church meetings Provide messages and information about physical activity and healthy eating that Pastors can talk about from the pulpit (refer to church resource binder for ideas) Encourage Pastor to be a role model by wearing the pedometer provided by the FAN program and encourage him/her to talk about it often

FAN D&I, the product of a series of CBPR projects (Wilcox et al., 2010; Wilcox et al., 2013), continued the collaborative partnership approach. As reflected in the FAN D&I logic model (see Table 2), FAN D&I collaborated with Fairfield County community organizations to identify and train Community Health Advisors (CHAs). The CHAs, in turn, trained and provided technical assistance to church committees, known as FAN Committees, who carried out FAN in their local churches (Sharpe et al., 2018). FAN implementation was hypothesized to result in changes in organizational practices, which would, in turn, have positive impacts on behavioral outcomes related to PA and HE.

Table 2.

FAN Diffusion and Implementation Logic Model for Implementation Monitoring

	Train the Trainer Activities	CHA Activities	Church Committee Activities	Implementation outcomes	Organizational outcomes	Individual outcomes & Impacts
Chain of events logic model	With the guidance of Fairfield Community Coordinating Council and in collaboration with Fairfield Behavioral Health Services, the USC FAN Team will train the Community Health Advisors (CHA) who will develop knowledge, confidence & skills and will then provide…	training, TA & follow up consultation to church committees and pastors to facilitate developing knowledge, confidence, and skills among FAN Committee members to create a health-promoting church environment which, with active pastor support, will result in…	FAN committees holding a FAN Kick-off event, holding monthly meetings to plan FAN activities, assessing their church environments, and developing a plan and budget to promote healthy eating and physical activity in their churches, which will be followed by	FAN committees delivering/installing core FAN elements (Table 3) in their churches in collaboration with other church ministries which will result in…	organizational change: enhanced 1) opportunities for physical activity (PA) and eating fruit & vegetable (FV); 2) positive characteristics of opportunities for PA & FV; 3) social structures and policies supporting PA & FV; and 4) supportive media and cultural messages which will result in…	enhancing physical activity and healthy eating self-efficacy, which will result in improvements in physical activity and eating fruits and vegetables
Measurement component/Purpose	Process: Dose delivered & received-CHA To ensure CHAs have knowledge, confidence, and skills needed to provide church FAN committee training and follow up TA	Process: Dose delivered & received and reach-FAN committees To ensure FAN committees CHAs have knowledge, confidence, and skills needed to carry out FAN in their churches	Process: Dose received-FAN committees To ensure FAN committees carry out recommended actions to engage congregants and develop structures needed for implementation	Process: Fidelity & Completeness Assess extent to which FAN committees implemented FAN in their churches	Outcomes: Organizational change To assess organizational uptake of health-promoting church elements	Outcomes: Individual change & Health impacts To assess outcomes and impacts at the individual behavior level
Measures and time frame	Staff records & documentation Training observation form Training attendance log Training evaluation & self-rating (CHAs) Time frame: at CHA training	Staff records & documentation Training observation form Training attendance log Training evaluation & self-rating (FAN committee members) Time frame: at FAN committee training	Staff records & documentation TA call log Plan review Time frame: during 12 months following FAN committee training	FAN coordinator interviews Time frame: Baseline (before training/random assignment), 12 months for implementation (& 24 months for sustainability)	Pastor interviews Time frame: Baseline (before training/randomization), 12 months for organizational change (& 24 months for sustainability)	Congregant surveys: Self-efficacy for HE and PA scales Self-reported MVPA Fruit and Vegetable intake Time frame: 12 months

Each FAN Committee attended a one-day training, created and submitted a plan and budget for how program components would be implemented in their church and worked together to implement the program in their church. During the training, committees went through an active “assessment and planning” process organized along Cohen’s structural model of health behavior (Cohen et al., 2000). Using an interactive workbook, CHAs guided churches through an assessment of current activities and a planning process to select ways to add, enhance, or expand them for each component of the model. The budget ($300 for smaller churches and $500 for larger churches) was for materials that would support FAN implementation. The funds were from the research grant that supported the project and were an incentive to participate in the program and the evaluation. After the plan and budget were reviewed and approved by research staff, committees implemented the plan in their church. All churches were asked to implement a core set of activities (see Table 1), with a great deal of flexibility in the specific approaches they used to address each of the structural factors (Wilcox et al., 2018).

FAN D&I Evaluation, Design, and Data Sources

The FAN D&I comprehensive evaluation was designed around the RE-AIM framework (Glasgow, Vogt, & Boles, 1999). Wilcox et al., (2018) reported reach, adoption, and effectiveness. The process evaluation reported in this paper focused on the implementation component of RE-AIM, guided by a comprehensive process evaluation framework and organized by a logic model (see Table 2) (Saunders et al., 2016). The focus of this paper is implementation fidelity of the elements of the FAN intervention that were installed by the church FAN committees.

Phase 1 of the FAN D&I study, previously reported (Wilcox et al., 2018), was a group randomized trial in which churches from a medically-underserved and rural county were invited to participate in the study. Churches were randomly assigned to be trained immediately (n=39) or 12 months later (n=20); these groups served as the intervention and control conditions, respectively. In collaboration with a county agency and council of community-level organizations, all churches in the county were identified. Eligible churches (located in the target county, had at least 20 members, willing to accept random assignment to training group) were invited to participate in FAN and those interested were enrolled. Detailed recruitment procedures have been reported previously (Wilcox et al., 2018).

This community-based study was designed and powered with a randomization ratio of 2:1 (2 intervention for every 1 control church). This design, which increased the likelihood that a church would be randomized to receive the intervention, made participating in the study more appealing to the church community. It also allowed us to use more resources in evaluating aspects of the intervention and implementation since a larger number of churches received the intervention. Churches were randomly assigned to condition after baseline telephone interviews with all pastor-designated FAN church contacts (FAN Coordinators) were conducted,. Baseline surveys assessed current church practices related to PA and HE. Telephone interviews using the same instrument and items were re-administered at 12 months to assess implementation of PA and HE activities in both intervention and control churches.

Behavioral outcomes (PA and HE self-efficacy, proportion active, proportion physically inactive, and FV intake) were assessed in all churches at the individual level with church members at 12-months only using self-completed surveys (Wilcox et al., 2018).

The study protocol was reviewed by the University of South Carolina Institutional Review Board and granted exempt status.

Data Collection Procedures

Baseline and 12-month telephone interviews with the FAN Coordinator were conducted by the Survey Research Laboratory (SRL) at the University of South Carolina. The research team provided the SRL a list of the FAN Coordinators’ names and telephone numbers. Prior to data collection, interviewers and interviewing supervisors at the SRL received specialized training. In addition, many of the interviews were monitored to ensure that instructions were being followed. The SRL’s computer-aided telephone interviewing system was used to complete the interviews.

Baseline interviews with FAN Coordinators were conducted from 9/2/2015 to 10/28/2015. All 59 FAN Coordinators from enrolled churches completed the interview (100% response rate). Interviews lasted 19 minutes on average. Of the churches randomized to the intervention group, 92% (36/39) of FAN Coordinators attended training. Three intervention churches did not attend training, and one additional intervention church withdrew after training but prior to 12-month interviews. One control church withdrew after randomization but prior to the 12-month interviews, and one control church completed the 12-month interview but did not attend the delayed training. Twelve-month interviews were conducted from 9/6/2016 to 11/3/2016. All 54 FAN Coordinators (35 intervention, 19 control) completed the interview (100% of those attempted; 91.5% of those randomized). Interviews lasted, on average, 29 minutes.

For church member data collection to assess study outcomes, data collectors who were blind to intervention assignment visited 54 churches (all but one visit took place on a Sunday) 8 to 12 months after the training of early intervention churches and administered the surveys immediately after the worship service. The data collectors distributed anonymous, 7-page, self-administered questionnaires with the offer of interviewer administration and/or assistance. To calculate survey response rate, data collectors counted the number of adults present at the worship service. Pastor-reported attendance was used in the six instances when data collectors were not invited to the worship service. Questionnaires were completed by 1,423 attendees; 115 were not used because they were missing a covariate, leaving a sample of 1308. Across churches, an estimated 71% of church attenders, completed the questionnaire (Wilcox et al, 2018).

Data collection tools and criteria for evidence of implementation

Implementation.

Measures for implementation of core components of FAN (implementation fidelity) were based on the conceptual model guiding the intervention (see logic model in Table 2) and were adapted from the implementation measures used in the prior FAN study (Wilcox et al, 2010; Wilcox et al, 2013). The FAN D&I Evaluation Committee engaged in a systematic and iterative review process of all FAN D&I measures to ensure consistency with the survey items and the conceptual model. In addition, all instruments were reviewed and approved by the community partners to ensure clarity and local appropriateness.

As described in Table 3, the four components for PA (guidelines for PA, opportunities for PA, messages about PA, and pastor support for PA) were assessed with 11 items. The six core components for HE (guidelines for HE-fruits, guidelines for HE-vegetables, opportunities for HE-fruits, opportunities for HE-vegetables, messages about HE, and pastor support for HE) were assessed with 9 items. Each core component was measured with 1 to 4 items, which were rated on a 4-point Likert scale. Mean scores were calculated for multi-item scales. We set the criteria for evidence of acceptable implementation a priori at 3 or 4 (the two highest response categories).

Table 3.

FAN D&I FC Implementation Measures

Conceptual Model Construct	Variable	Number of items	Response scale	Sample items
Physical Activity
Social structures	Guidelines for physical activity	1	4 to 1 (“Yes, it is fully in place” to “No, not at this time)	In the past year, was a church guideline put into place that stated physical activity would be incorporated into church meetings and events?
Availability/accessibility of products	Opportunities for physical activity	4	4 to 1 (“almost all of the time” to “not at all”)	In the past year, how often were opportunities to be physically active incorporated before, during, or after worship service?
Media and cultural messages	Messages about physical activity	4	4 to 1 (“About weekly” to “rarely or never”)	In the past year, how often was new information about physical activity posted on a bulletin board in your church?
	Pastor support for physical activity	2	4 to 1 (“About weekly” to “rarely or never”)	In the past year, how often did the Pastor include messages about physical activity during church services?
Healthy Eating
Social structures	Guidelines for healthy eating-fruits	1	4 to 1 (“Yes, it is fully in place” to “No, not at this time)	In the past year, was a church guideline put into place that stated fruits or fruit dishes will be included in all meeting and events when food is served?
	Guidelines for healthy eating-vegetables	1	4 to 1 (“Yes, it is fully in place” to “No, not at this time)	In the past year, was a church guideline put into place that stated vegetables or vegetable dishes will be included in all meeting and events when food is served?
Availability/accessibility of products	Opportunities for healthy eating-fruits	1	4 to 1 (“almost all of the time” to “not at all”)	In the past year, how often were fruits or fruit dishes made available to church members at church functions that included food?
	Opportunities for healthy eating-vegetables	1	4 to 1 (“almost all of the time” to “not at all”)	In the past year, how often were vegetables or vegetable dishes made available to church members at church functions that included food?
Media and cultural messages	Messages about healthy eating	4	4 to 1 (“About weekly” to “rarely or never”)	In the past year, when church bulletin or bulletin inserts were given to church members, how often were messages about healthy eating provided?
	Pastor support for healthy eating	1	4 to 1 (“About weekly” to “rarely or never”)	In the past year, how often did the Pastor include messages about healthy eating during church services?

Outcome measures.

Outcome measures have been previously reported and will be summarized here (Wilcox et al., 2018). We administered a 5-item measure of self-efficacy for overcoming common barriers to PA (Marcus, Eaton, Rossi, & Harlow, 1994) and an 8-item measure of self-efficacy for eating FV in various situations (Resnicow et al., 2004). Responses ranged from 1 (not at all confident) to 7 (very confident). We computed a mean score for each scale.

We calculated the proportion of members physically inactive (<10 mins/wk of PA) and regularly physically active (≥150 mins/wk of moderate PA, ≥ 75 mins/wk of vigorous PA, or an equivalent combination) using six questions (3 moderate PA, 3 vigorous PA) from the 2010 Behavioral Risk Factor Surveillance System (BRFSS) PA module (CDC, 2017). For fruit intake, we asked members to report “About how many cups of fruit (including 100% pure fruit juice) do you eat or drink each day?” and included a parallel question for vegetables. Sample serving sizes for each were included. This measure was sensitive to change in other faith-based studies (Resnicow et al., 2004).

Data analysis

We used three analytic approaches to: 1) test differences in implementation between intervention and control churches; 2) assess church level implementation in intervention churches for PA and HE core components; and 3) examine the impact of level of implementation on project outcomes. We tested differences in implementation between early intervention and delayed intervention (control) churches over time with repeated measures regression models, controlling for church size. We also calculated effect sizes (Cohen’s d) for each variable.

To assess individual church level implementation, each intervention component score at the 12-month follow-up was regressed on its baseline score. For each intervention church, the reported 12-month score was expressed as a deviation above or below the expected value given that church’s baseline score. Positive deviations from expectation were interpreted as higher implementation. There were two criteria used to classify a church as a higher implementing church: 1) a score of 3.0 or higher at 12 months, and 2) a deviation greater than 0 or a score of 4.0 for at least three of the four components at 12 months. Therefore, a church was classified as “high implementing” by either increasing implementation relative to that church’s baseline level or by sufficiently high implementation at 12 months in absolute terms, regardless of baseline score. For this analysis, FV opportunities were combined, and FV guidelines were combined, reducing the number of core HE components from six to four.

To examine the impact of level of implementation on outcomes, we compared member-reported outcome variables among control, higher implementing intervention, and lower implementing intervention churches at 12 months using multi-level post-test only regressions models, controlling for members’ age, gender, and education, as well as church size and predominant race of congregation with clustering of members within churches accounted for. There were separate models for PA and HE implementation (FAN Coordinator-reported) and outcomes (church member-reported). We also calculated effect sizes (Cohen’s d or Odds Ratio) for each variable.

Results

Comparing intervention and control churches

Coordinators from 54 churches (35 intervention, 19 control) completed interviews at both time points (94% African American, 96% women, 60±9 years of age; 37% with some college or greater education; 54% obese). There were no significant baseline differences between the two groups on any implementation variables (data not shown). Intervention compared to control churches reported significantly higher implementation at 12 months in opportunities for PA, guidelines for PA, HE-fruits and HE-vegetables; messages for PA and HE, and pastor support for PA and HE. The magnitudes of these effects were large (d=0.7 to d=1.6, (p values .02 to <.001). There were no differences between the two groups for opportunities for eating vegetables, which was high in both groups at baseline, nor in opportunities for fruit, although the magnitude of the group difference was moderate and favored intervention churches (d=0.5, p=.10) (Table 4).

Table 4.

LS Means and Standard Errors, Effect Sizes and p-values for Repeated Measures Analysis for FAN Contact-reported Healthy Eating (HE) and Physical Activity (PA) Implementation at Baseline and 12 Months in Control and Intervention Churches

	Control		Intervention		Effect size	p*
	Baseline	12 months	Baseline	12 months
Implementation variable	LSM (SE)	LSM (SE)	LSM (SE)	LSM (SE)
Physical Activity (PA)
Guidelines PA	2.1 (0.23)	2.0 (0.23)	2.0 (0.17)	2.9 (0.17)	1.2	.002
Opportunities PA	1.8 (.15)	1.6 (0.15)	1.8 (0.11)	2.7 (0.11)	1.5	<.001
Pastor Support for PA	1.2 (0.15)	1.4 (0.15)	1.4 (0.11)	2.5 (0.11)	1.6	<.001
Messages PA	1.3 (0.15)	1.4 (0.15)	1.6 (0.12)	2.7 (0.12)	1.5	<.001
Healthy Eating (HE)
Guidelines HE fruit	2.2 (0.22)	2.2 (0.22)	2.2 (0.17)	3.2 (0.17)	1.2	<.001
Guidelines HE vegetables	2.2(0.25)	2.4 (0.26)	2.4 (0.19)	3.3 (0.19)	0.7	.020
Opportunities HE fruit	3.2 (0.17)	3.3 (0.17)	3.1 (0.13)	3.6 (0.13)	0.5	.101
Opportunities HE vegetables	3.8 (0.13)	3.9 (0.13)	3.7 (0.10)	3.8 (0.10)	0.0	.808
Pastor Support HE	1.4 (0.19)	1.6 (0.19)	1.7 (0.14)	2.8 (0.14)	1.1	<.001
Messages HE	1.6 (0.17)	1.5 (0.17)	1.8 (0.12)	2.9 (0.12)	1.5	<.001

^*Repeated measures analysis comparing 12-month implementation in intervention and control churches adjusting for church size

Church level implementation

Church level implementation was assessed using criteria set a priori and also considered baseline levels of implementation. There was church level variation in implementation among the intervention churches for both HE and PA (see supplemental files for detailed tables). Twenty-one churches (60%) implemented PA guidelines; 13 (37%) churches implemented PA opportunities; 13 (37%) churches implemented PA messages; and 13 (37%) churches implemented PA pastor support. Ten (29%) churches implemented at least three of the four PA components (that is, 10 churches were “higher” implementers in PA).

Based on criteria set a priori, 21 (60%) churches implemented HE guidelines; 25 (71%) churches implemented HE opportunities; 18 (51%) of churches implemented HE messages; and 18 (51%) churches implemented pastor support. Twenty (57%) churches implemented at least three of the four HE components (that is, 20 churches were “higher” implementers in HE).

Examining impact of level of implementation on project outcomes

Members of higher implementing intervention, lower implementing intervention, and control churches did not report significantly different PA self-efficacy or FV self-efficacy (Table 5). For behavioral outcomes, members of lower implementing intervention churches reported a significantly lower percentage of physically inactive participants compared to control churches. There were no group differences on the other behavioral outcomes of percentage meeting the PA guidelines for FV intake (Table 5).

Table 5.

Comparison of Intervention Higher Implementers, Intervention Lower Implementers, and Control Churches on Member-reported Physical Activity and Healthy Eating-Related Variables, 8 to 12 months after Intervention Training

Outcome	Intervention Higher Implementer LSM (SE)	Intervention Lower Implementer LSM (SE)	Control LSM (SE)	Effect size (d) Early-High vs. Control or Odds Ratio (95% CI)	Effect size (d) Early-Low vs. Control or Odds Ratio (95% CI)	F	p
Self-efficacy
Physical activity self-efficacy	3.7 (0.21)	3.5 (0.19)	3.3 (0.17)	0.1876	0.0913	2.13	.13
Fruit and vegetable self-efficacy	4.4 (0.20)	4.4 (0.19)	4.3 (0.17)	0.1038	0.0946	.82	.45
Behavioral outcomes
Fruit and vegetable intake, cups/day	3.6 (0.29)	3.6 (0.28)	3.5 (0.25)	0.0768	0.0507	.46	.63
Fruit intake, cups/day	1.6 (0.17)	1.6 (0.17)	1.5 (0.15)	0.0769	0.0501	.37	.69
Vegetable intake, cups/day	2.0 (0.16)	2.0 (0.15)	1.9 (0.13)	0.0740	0.0467	.58	.58
Percentage physically inactive	16.1 (5.40)	11.4 (3.53)	21.2 (4.95)	0.717 [0.389, 1.320]	0.483 [0.292, 0.797]	4.27	.02
Percentage meeting physical activity recommendations	68.3 (6.72)	73.1 (5.47)	68.8 (5.36)	1.019 [0.656, 1.583]	1.271 [0.886, 1.824]	1.03	.37

Note. Models adjusted for age, gender, education, church size, church clustering, and predominant race

Discussion

As anticipated, implementation of the FAN core elements, except for opportunities for eating FVs, was significantly greater in early intervention compared to control churches. These results are also consistent with the FAN D&I outcomes, reported by church congregants (Wilcox et al., 2018) and with the results of the prior FAN trial (Wilcox et al., 2013), demonstrating replication and consistency.

Opportunities for eating FVs were high at baseline in both groups, making increases over time and group differences unlikely at 12 months. Despite these initially high values, the difference between control and intervention groups was in the expected direction for opportunities for eating fruits and was moderate in magnitude, suggesting we only had power to detect large differences.

Individual church variation in implementation is to be expected and is commonly reported across a variety of field settings; implementation within a given project may vary considerably with 20–40% differences among sites common (Durlak & DuPre, 2008 Accordingly, some individual churches reported implementation at higher levels of fidelity than others, and we were able to assign churches to higher and lower implementing categories based on criteria set a priori using novel methodology in which 12-month implementation was contrasted with expected 12-month implementation considering baseline levels of practice.

There was lower overall implementation of PA (10 or 29% higher implementers) compared to HE (20 or 57% higher implementers) core components. This may be due in part to less pre-existing church infrastructure for PA relative to HE (Brand & Alston, 2017). Most churches have infrastructure in place for providing food (e.g., kitchens for preparing and facilities for serving), whereas the structures and supports for PA must be developed, which would likely require more time (Kaczynski et al., 2018). This is also consistent with the observation that baseline HE opportunities were high in both conditions. In separate papers, we are investigating implementer and church-related factors to understand the variability in implementation and are also examining qualitative data on implementation barriers and facilitators to identify elements of the FAN intervention and implementation approach that could be changed to facilitate higher levels of implementation.

Lower, but not higher, implementers in the early churches had significantly more members who were physically inactive compared to control churches. Therefore, the results do not support a “dose response” effect of implementation on this behavioral outcome. We made a priori decisions about what constituted higher implementation; nevertheless, it is not known how much implementation is required to effect change or whether implementation operates via a dose response or a threshold effect. Future research is needed to address these issues.

Limitations and Strengths

Limitations of this study include self-reported implementation from a single data source (FAN coordinators) and self-reported individual behavior (congregants), although the results are congruent with those independently reported by congregants in the churches. The implementation measures were developed through a systematic process to ensure consistency with the FAN conceptual model and cultural appropriateness; nevertheless, we did not assess validity and reliability formally. The FAN training also emphasized decreasing unhealthy fats and sodium and increasing whole grains, but due to respondent burden, we were not able to assess each of these components. As a result, we do not know how FVs were prepared. It is possible that intervention churches prepared these items with less fat and sodium than control churches. We did not assess behavior at baseline, so it is possible that some behavioral differences were present at that point. Furthermore, policy, systems, and environmental change and their impacts on individual behavior may take longer than the 12-month duration of this project. Strengths include a quantitative implementation assessment using fidelity criteria set a priori, a strong study design (i.e., baseline assessment prior to assignment to condition, random assignment to conditions, and assessments conducted in both intervention and control groups), a culturally appropriate partnership approach to the intervention, and the relatively large number of participating churches.

FAN D&I is one of the few studies in faith-based settings that have assessed and reported fidelity; according to a recent review in faith settings, only 9% report implementation fidelity (Yeary et al., 2012). Furthermore, our rigorous process evaluation design that included baseline assessments prior to random assignment, random assignment to condition, and assessments in intervention and control groups, enabled us to assess with confidence greater levels of implementation in intervention compared to control churches and church level implementation considering baseline level of practice. We found significantly higher levels of implementation of PA opportunities, PA and HE guidelines, PA and HE messages, and PA and HE pastor support in intervention versus control churches, as well as church-level variation in PA and HE implementation. High and low implementers did not differ in proportion of church members physically inactive, although low implementers compared to control had fewer members physically inactive.

Implications for Theory, Policy, and Practice

The methodological implications of this study include the importance of assessing implementation fidelity and using a rigorous evaluation design. To the best of our knowledge this is one of the few implementation monitoring studies to use a control group as recommended (Yeary et al., 2012) and the first to incorporate a measure of baseline practices in assessing 12-month implementation outcomes. This contribution is important because there are likely secular trends that could affect church practices as well as PA and HE and behaviors of church congregants; our rigorous design helps address potential threats to internal validity and increases confidence in the findings. For example, without the combined baseline assessment, random assignment, and control group comparisons, it is difficult to ascertain what is happening with the 12-month opportunities for HE. With post-test only in the intervention group, we could conclude FAN has a favorable impact; with a comparison group and post-test only, we might conclude FAN did not have a favorable impact.

Instead, we know that HE baseline opportunities were very high in both groups at baseline and were maintained over time in the control group for FVs and maintained in the intervention group for vegetables but increased for fruit. Clearly, it is important to use a rigorous evaluation design in implementation monitoring. Furthermore, given the design and our confidence in our results showing high levels of FAN D&I implementation, it appears that FAN is well-poised for broader dissemination in faith-based settings.