Evaluation of organizational capacity in the implementation of a church-based cancer education program

Abstract

Implementation evaluations have increasingly taken into account how features of local context help determine implementation outcomes. The purpose of this study was to determine which contextual features of organizational capacity led directly to the RE-AIM Framework implementation outcomes of intervention reach and number of days taken to implement, in an implementation trial of a series of cancer education workshops conducted across 13 African American churches in Maryland. We used a configurational approach with Coincidence Analysis to identify specific features of organizational capacity that uniquely distinguished churches with implementation success from those that were less successful. Aspects of organizational capacity (e.g., congregation size, staffing/volunteers, health ministry experience) were drawn from an existing measure of church organizational capacity for health promotion. Solution pathways leading to higher intervention reach included: having a health ministry in place for 1–4 years; or having fewer than 100 members; or mid-size churches that had conducted health promotion activities in 1–4 different topics in the past 2 years. Solution pathways to implementing the intervention in fewer number of days included: having conducted 1–2 health promotion activities in the past 2 years; or having 1–5 part-time staff and a pastor without additional outside employment; or churches with a doctorally prepared pastor and a weekly attendance of 101–249 members. Study findings can inform future theory, research, and practice in implementation of evidence-based health promotion interventions delivered in faith-based and other limited-resource community settings. Findings support the important role of organizational capacity in implementation outcomes in these settings.

Implementation evaluations have increasingly considered how context helps determine implementation outcomes (Durlak & DuPre, 2008; Gingiss et al., 2006; Kelly et al., 2004; Norton, 2012; Riley et al., 2003; Woltmann et al., 2008). Organizational factors have been explicitly incorporated into multiple implementation science theories and/or models, including the Consolidated Framework for Implementation Research (CFIR) (Damschroder et al., 2009), which features the role of inner setting factors such as structure, communication, and readiness. Similarly, the Practice, Robust Implementation and Sustainability Model (PRISM) (Feldstein & Glasgow, 2008) highlights the importance of infrastructure and organizational characteristics, which along with other factors, impact implementation outcomes. These theories and others (Aarons et al., 2012; Meyers et al., 2012) have guided a rich wave of recent studies that examine the role of context in implementation outcomes.

One particular aspect of context of special interest to implementation science is organizational capacity. Though sometimes conflated with organizational readiness, organizational capacity is a discrete construct (Rabin & Brownson, 2012; Weiner et al., 2008). Organizational capacity is defined as the observable, structural aspects of an organization, assessed at the organizational level (Rabin & Brownson, 2012; Tagai et al., 2018), and can include things such as financial and material resources. Organizational capacity has been shown to be relevant to a number of implementation outcomes. Findings from 81 studies included in a meta-analysis indicate that organizational capacity (e.g., staffing and funding) has critical implications for the process of implementation (Durlak & DuPre, 2008). Financial provisions and leadership support were positively associated with intervention adoption in healthcare settings (Norton, 2012). Organizational capacity (e.g., staffing and funds) has been linked to increased implementation fidelity in school-based tobacco prevention programs (Gingiss et al., 2006). Staff turnover has been linked to decreased program implementation (i.e., fidelity and penetration) in mental healthcare settings (Woltmann et al., 2008). Research suggests adequate staffing positively affects the number of heart health promotion strategies implemented in public health agencies (Riley et al., 2003), as well as number of program sessions delivered and resulting changes in dietary behavior in health education programs (Kelly et al., 2004).

The role of organizational capacity is likely to be particularly salient when interventions are implemented in limited resource settings, such as community settings outside the healthcare system. These are settings that regularly engage in health promotion activities, but the primary mission, and corresponding resource allocation, is something other than health promotion. A notable example here is the health promotion work done in the context of churches, which have served key community support and resource functions in African American communities in America since the time of slavery (Lincoln & Mamiya, 1990). Allen and colleagues (Allen et al., 2015) recognized the importance of church organizational capacity when they designed and implemented an organization-level intervention to increase capacity of Catholic churches serving Latinos to provide evidence-based cancer control interventions. The team concluded that many Catholic parishes have the capacity to implement evidence-based cancer control interventions if they are adapted to fit the setting (Allen et al., 2016). Guided by CFIR, a recent study examined predictors of 12-month implementation of the Faith, Activity, and Nutrition intervention in United Methodist churches (Wilcox et al., 2021). The authors reported that intervention coordinator ratings of inner setting factors (e.g., communication, culture, organizational rewards) were predictive of their implementing the intervention’s core components.

The Present Study

Both the recent growing attention to the role of organizational factors in implementation science research and the observation that African American churches vary greatly in their organizational capacity (Tagai et al., 2018), lead to the question of what are the organizational determinants of successful implementation of evidence-based interventions delivered in these settings. The purpose of the present study was to determine which individual and combinations of organizational capacity indicators were co-present with key implementation outcomes, identified based on the RE-AIM Framework. These outcomes included intervention reach, defined as the proportion of eligible individuals participating in the intervention (Glasgow et al., 1999) and the number of days taken to implement a series of three cancer educational workshops, in an implementation trial conducted in African American churches in central Maryland. Because churches vary so greatly in their organizational capacities and often serve as health promotion settings even in the face of limited resources, it is important to identify the organizational conditions that can facilitate successful intervention implementation.

We used configurational analysis to identify determinants of implementation success. Configurational analysis is a mathematical, cross-case approach that operates within a regularity framework (Baumgartner & Falk, in press) and applies Boolean algebra and formal logic to identify a “minimal theory,” i.e., the key difference-making conditions that uniquely distinguish one group from another. Configurational analysis, also known as Configurational Comparative Methods, incorporates both Qualitative Comparative Analysis (QCA) and more recently Coincidence Analysis (CNA), the approach used here. Configurational analysis searches for necessary and sufficient conditions for an outcome to appear and is well-suited to identify both Boolean conjunctivity (when the joint appearance of several conditions yields an outcome) and disjunctivity (when multiple pathways lead to the same outcome) (Furnari et al., 2020; Palinkas et al., 2019; Ragin, 2014). Because configurational analysis draws on Boolean algebra (as opposed to linear algebra and probability), it does not require large sample sizes and can be applied with small-n studies. Unlike traditional variable-oriented methods, configurational analysis retains a persistent link to individual cases, applying formal logic to develop models identifying the specific bundles of conditions that distinguish cases with an outcome of interest from those without. Configurational analysis in general–and Coincidence Analysis in particular–has appeared across a wide variety of health-related implementation contexts in the published literature since 2020 (Cohen et al., 2021; Coury et al., 2021; Hickman et al., 2020; Miech et al., 2021; Petrik et al., 2020; Whitaker et al., 2020; Yakovchenko et al., 2020).

Methods

Project HEAL Intervention

The intervention platform for this secondary analysis is “Project HEAL” (Health through Early Awareness and Learning) (Holt et al., 2018; Holt et al., 2014; Santos et al., 2017; Santos et al., 2014; Scheirer et al., 2017). The Project HEAL intervention trains lay members of African American churches as Community Health Advisors, who then conduct a series of three cancer educational workshops in their churches on breast, prostate, and colorectal cancer, with an emphasis on early detection. Details on the intervention are described elsewhere (Holt et al., 2014). The Project HEAL intervention was implemented in 14 African American churches in Prince George’s County, Maryland. The Community Health Advisors complete a 13-module training and pass a cancer knowledge examination with a score of 85% or greater before they become certified to implement the workshop series. The cancer educational workshops are designed to both reinforce and build on each other and are intended to be delivered within a 60-day period of time (workshop 1, workshop 2 at 30 days later, workshop 3 at 30 days later), which makes the number of days to complete the workshop series a primary implementation outcome. The Community Health Advisors use a PowerPoint slide deck with direct spiritual references to present the workshop material, and they distribute project-specific print educational booklets to attendees. The Project HEAL workshops have been shown to result in significant baseline to 24-month increases in breast, prostate, and colorectal cancer knowledge, as well as significant increases in reports of fecal occult blood test and colonoscopy, mammography maintenance among women, and digital rectal exams in men (Holt et al., 2018).

Measures

Organizational Capacity

Indicators of church organizational capacity for health promotion were utilized from the Faith-Based Organization Capacity Inventory (FBO-CI) (Tagai et al., 2018). The FBO-CI was developed to assess organizational capacity specific to faith-based organizations and was grounded on theoretical work by Greenhalgh and colleagues (Greenhalgh et al., 2004). The instrument contains three subscales, including staffing and space (e.g., number of full- and part-time staff), health promotion experience (e.g., presence of health ministry; health promotion activities conducted in the past 2 years), and external collaboration (e.g., collaborations with outside organizations; participation in research). Internal reliability of the subscales ranged from 0.67 to 0.83 (Tagai et al., 2018). Select individual capacity indicators (items) were coded as multi-value ordinal factors with up to 4 possible levels for analysis based on their response scales and distributions (see Table 1).

Table 1.

Indicators of Church Organizational Capacity and Sample Distribution in N=13 Churches

FBO-CI Construct	Item	Levels	Coding	N (%)
Staffing/space	Membership size	3	0–100	4 (30.8)
			101–499	7 (53.8)
			500+	2 (15.4)
	Weekly attendance	3	0–100	4 (30.8)
			101–249	6 (46.2)
			250+	3 (23.1)
	Pastor outside employment	2	no	11 (84.6)
			yes	2 (15.4)
	Pastor education	3	other	3 (23.1)
			masters	5 (38.5)
			doctorate	4 (30.8)
	Building ownership	2	no	3 (23.1)
			yes	9 (69.2)
	Number full-time staff	3	0	3 (23.1)
			1	5 (38.5)
			2+	5 (38.5)
	Number part-time staff	3	0	4 (30.8)
			1–5	4 (30.8)
			6+	5 (38.5)
	Number volunteers	3	0	0 (0.0)
			1–25	2 (15.4)
			26–50	4 (30.8)
			51+	7 (53.8)
Health Promotion Experience	Current health ministry	2	no	4 (30.8)
			yes	9 (69.2)
	Number health areas past 2 years	4	0	0 (0.0)
			1–4	5 (38.5)
			5–8	3 (23.1)
			9+	5 (38.5)
	Number health activities past 2 years	4	0	0 (0.0)
			1–2	3 (23.1)
			3–4	5 (38.5)
			5+	5 (38.5)
	Years of health ministry existence	4	0	4 (30.8)
			1–4	3 (23.1)
			5–10	4 (30.8)
			11+	2 (15.4)

Note. FBO-CI = Faith-Based Organization Capacity Inventory. Numbers might not sum due to missing data. External collaboration subscale items were not significant in any analysis and are not shown.

Implementation Outcomes

Two primary implementation outcomes from the Project HEAL intervention were reach and time to intervention completion. These were selected because of their alignment with the RE-AIM (Reach, Efficacy, Adoption, Implementation, and Maintenance) Framework (Glasgow et al., 1999), which was used to guide the evaluation, and there was considerable between-church variability in the data, suggesting both uneven and in some cases sub-optimal implementation with opportunity for future growth. Reach was calculated as the proportion of church members who enrolled in the intervention divided by an estimate of the total number of eligible members. Reach was dichotomized into low and high on the basis of a median split (10%−29% = low; 30%−78% = high). Time to completion was defined as the number of days that the church took to implement all three cancer educational workshops in the series and was also dichotomized into low and high based on a median split and on the recommended implementation time frame from the study protocol (28–60 days = more timely completion; 61–84 days = less timely completion).

Factor Selection

The original dataset had over 50 factors based on the FBO-CI indicators described above. To reduce this data, we implemented a configurational approach to factor selection described in detail elsewhere (Cohen et al., 2021; Coury et al., 2021; Hickman et al., 2020; Petrik et al., 2020; Yakovchenko et al., 2020). Briefly, we began by using the “minimally sufficient conditions” (msc) function within the R package “cna” to comprehensively scan the full original dataset in order to identify configurations of conditions with strong connections to the outcomes (e.g., reach and time to completion). This process exhaustively considers all one-, two- and three-condition configurations instantiated in the dataset, assesses each configuration against a prespecified consistency threshold, retains all configurations that satisfy this criterion, and then generates a “condition table” to list and organize the Boolean output. In a condition table, rows contain all configurations of conditions that meet a specified consistency level while column variables include outcome, conditions, consistency, and coverage. We started the msc routine by specifying a consistency threshold of 100%; if no configurations met this threshold, we iteratively lowered the specified consistency level by 5 points (e.g., from 100% to 95%, etc.) and repeated the process to generate a new condition table. We continued lowering the consistency threshold until there were at least two potential configurations of church-level conditions that met the specified consistency level. Using this approach, we inductively analyzed the entire dataset and used the condition table output to identify a subset of candidate factors for model development in the next step of configurational analysis.

Model Development

We next developed models for reach and time to workshop completion by iteratively using model-building functions within the “cna” software package in R. We assessed models based on their overall consistency and coverage as well as potential model ambiguity (when more than one model satisfies the specified consistency and coverage thresholds and explains the outcome of interest). We selected a final model based on the same criteria of overall consistency and coverage, with no model ambiguity. The Coincidence Analysis package (“cna”) in R (Ambühl et al., 2021), R (version 3.5.0), R Studio (version 1.1.383) and Microsoft Excel were used to support the analyses. One church did not have a recorded reach outcome and was thus dropped from the analysis, resulting in an analytic sample of 13 churches.

Results

Table 1 provides the organizational capacity indicators, their codings, and the sample distribution for each indicator. Over half of the churches in the sample were in the medium size range of 101–499 members on their rosters, most of the pastors did not have outside employment in addition to their role in the church, and both full- and part-time staff numbers were modest in number while the number of volunteers was considerably greater. Nearly three-fourths of the churches had an existing health ministry.

With regard to the outcome variables, reach varied from 10 to 78%, with a median of 29% and a mean of 33% (SD=0.23) of eligible participants enrolled in the intervention. Time to complete the 3-workshop series varied from 28 to 84 days, with a median of 59.5 days. These data reflected considerable between-church heterogeneity in the implementation outcomes.

Configurational Analyses

Table 2 shows the configurational model for the higher value of the reach outcome, with 30–78% of eligible members participating in the intervention. This model had three distinct solution pathways composed of just three factors. Churches that had greater intervention reach were those that had: 1) 1–4 years of health ministry experience; OR 2) fewer than 100 members; OR 3) 101–499 members AND had conducted health promotion activities in 1–4 different health topics in the previous 2 years. Configurational models have two main parameters of fit: consistency and coverage. Consistency measures how reliably a model yields an outcome, and is calculated by “the number of cases identified by a model that also have the outcome of interest” divided by “the total number of cases identified by the model.” This solution for higher reach had 100% consistency (6/6), as there were six cases identified by the model that also had the outcome (n=6) divided by six cases total that were identified by the model (n=6). Coverage measures the explanatory breadth of a model, and is calculated by “the number of cases identified by a model that also have the outcome of interest” divided by “the total number of cases that have the outcome of interest.” This solution had 100% coverage (6/6), as there were six cases identified by the model that also had the outcome (n=6) divided by six cases total that had the outcome of interest (n=6).

Table 2.

Configurational Model with Three Solution Pathways for Higher Value of Reach Outcome in N=13 Churches

Outcome: Reach	# Years health ministry history	Number of members	Number of members	# Health areas in past 2 years
High	5–10	<100	<100	9+
High	1–4	101–499	101–499	1–4
High	0	<100	<100	1–4
High	0	101–499	101–499	1–4
High	1–4	<100	<100	5–8
High	1–4	<100	<100	9+
Low	0	101–499	101–499	9+
Low	5–10	101–499	101–499	5–8
Low	5–10	500+	500+	5–8
Low	11+	101–499	101–499	9+
Low	5–10	101–499	101–499	9+
Low	0	500+	500+	1–4

Note. Reach = Percent of eligible members participating in the intervention: 10%−29% = Low; 30%−78% = High (based on median split). Shading is used for clarity in reporting the configurational analysis.

Table 3 shows the configurational model for the lower value of the reach outcome, with 10–29% of eligible members participating in the intervention. This model had three distinct solution pathways composed of just three factors. Churches that had lower intervention reach were those that had: 1) 500 or more members; OR 2) a medium membership size (101–499) AND had conducted health promotion activities in 9 or more different health topics in the previous 2 years; OR 3) a moderate (5–10 years) history of health ministry experience AND had conducted health promotion activities in 5–8 different health topics in the previous 2 years. As before, each solution pathway was sufficient by itself for the outcome of interest. This configurational model for lower reach likewise had both 100% consistency (6/6) and 100% coverage (6/6).

Table 3.

Configurational Model with Three Solution Pathways for Lower Value of Reach Outcome in N=13 Churches

Outcome: Reach	Number of members	Number of members	# Health areas in past 2 years	# Years health ministry history	# Health areas in past 2 years
Low	101–499	101–499	9+	0	9+
Low	101–499	101–499	5–8	5–10	5–8
Low	500+	500+	5–8	5–10	5–8
Low	101–499	101–499	9+	11+	9+
Low	101–499	101–499	9+	5–10	9+
Low	500+	500+	1–4	0	1–4
High	<100	<100	9+	5–10	9+
High	101–499	101–499	1–4	1–4	1–4
High	<100	<100	1–4	0	1–4
High	101–499	101–499	1–4	0	1–4
High	<100	<100	5–8	1–4	5–8
High	<100	<100	9+	1–4	9+

Note. Reach = Percent of eligible members participating in the intervention: 10%−29% = Low; 30%−78% = High (based on median split). Shading is used for clarity in reporting the configurational analysis.

Table 4 shows the configurational model for more timely completion of the 3-workshop series (28–60 days). This model had three distinct solution pathways composed of just five factors. Churches that took fewer days to complete the 3-workshop series were those that had: 1) 1–2 health activities in the previous 2 years; OR 2) 1–5 part-time staff AND the pastor did not have outside employment in addition to the church; OR 3) pastors who had a doctoral degree AND mid-sized (101–249) weekly attenders. This model similarly had both 100% consistency (6/6) and 100% coverage (6/6).

Table 4.

Configurational Model with Three Solution Pathways for More Timely Completion (MTC) of 3-Workshop Series in N=13 Churches

Outcome: # Days to complete 3 workshops	# Health activities in past 2 years	# Part-time staff	Pastor outside employment	Pastor education	Weekly member attendance
MTC	5+	0	No	Doctorate	101–249
MTC	5+	1–5	No	Doctorate	<100
MTC	3–4	0	No	Doctorate	101–249
MTC	1–2	6+	No	Masters	101–249
MTC	5+	1–5	No	Masters	250+
MTC	1–2	1–5	No	Masters	<100
MTC	1–2	6+	No	Other	101–249
LTC	3–4	6+	No	Masters	101–249
LTC	5+	0	No	Doctorate	250+
LTC	3–4	6+	No	Masters	250+
LTC	3–4	0	Yes	Other	<100
LTC	5+	1–5	Yes	Other	<100

Note. Number of days to complete workshop series: 28–60 days = More Timely Completion (MTC); 61–84 days = Less Timely Completion (LTC).

Shading is used for clarity in reporting the configurational analysis.

Table 5 shows the configurational model for less timely completion of the 3-workshop series (61–84 days). This model also had three distinct solution pathways composed of just five factors. Churches that took more days to complete the 3-workshop series were those that had: 1) pastors with outside employment in addition to the church; OR 2) conducted 3–4 health promotion activities in the previous 2 years AND had 6 or more part-time staff; OR 3) had 51 or more volunteers AND 250 or more members in weekly attendance. This configurational model also had both 100% consistency (6/6) and 100% coverage (6/6).

Table 5.

Configurational Model with Three Solution Pathways for Less Timely Completion (LTC) of the 3-Workshop Series in N=13 Churches

Outcome: # Days to complete 3 workshops	Pastor outside employment	# Health activities in past 2 years	# Part-time staff	# Volunteers	Weekly member attendance
LTC	No	3–4	6+	51+	101–249
LTC	No	5+	0	51+	250+
LTC	No	3–4	6+	51+	101–249
LTC	No	3–4	6+	51+	250+
LTC	Yes	3–4	0	26–50	<100
LTC	Yes	5+	1–5	26–50	<100
MTC	No	5+	0	51+	101–249
MTC	No	5+	1–5	1–25	<100
MTC	No	3–4	0	26–50	101–249
MTC	No	1–2	6+	51+	101–249
MTC	No	5+	1–5	26–50	250+
MTC	No	1–2	1–5	1–25	<100
MTC	No	1–2	6+	51+	101–249

Note. Number of days to complete workshop series: 28–60 days = More Timely Completion (MTC); 61–84 days = Less Timely Completion (LTC).

Shading is used for clarity in reporting the configurational analysis.

Discussion

The current study identified features of organizational capacity that can help explain key implementation outcomes in a trial that evaluated implementation of an evidence-based cancer control intervention delivered by volunteers in African American churches. Though organizational factors are clearly important to implementation success, to date there has been limited ability to examine these factors using traditional variable-oriented methods in small-n studies. Overall, there were several church organizational capacity factors consistently related to the implementation outcomes of intervention reach and the number of days that it took the churches to implement the full series of three Project HEAL cancer educational workshops.

Reach

Reach is a key outcome in implementation research and the first element of the RE-AIM Framework (Glasgow et al., 1999). Reach reflects the penetration of an intervention to impact its intended audience. Though in many cases, as in the current one, it is an estimate due to an uncertain denominator (Santos et al., 2017), data on reach is informative as an implementation outcome. Churches with better reach were those that had some, but not many, years of health ministry history. This reflects the idea that there may not be a linear, but rather a curvilinear, relationship between history of health promotion and better implementation outcomes. It is possible that in churches that had some experience, members viewed the Project HEAL intervention as novel or were curious or otherwise had an appetite for the information and were thus more inclined to engage than churches in which health ministry activities were commonplace. In the mid-size churches, reach was greater if they also had focused on just a few health topics in the past two years. Again, this may reflect a relative novelty of the Project HEAL workshops.

It is notable that reach was higher in the smaller congregations. This finding reflects the idea that bigger does not automatically equate to better when it comes to church size and the ability of its programming to serve the membership. The smaller churches may be more socially cohesive, with a fewer number of competing ministries and with most members attending a single event rather than having to choose between multiple concurrent offerings on the church calendar. It may have been easier for the Community Health Advisors to get the word out about the workshops in the smaller congregations due to the nature of the social network. In small congregations leaders other than the pastor (e.g., deacon/deaconess; first lady [pastors’s wife]) often can suggest the inclusion of an announcement in the church bulletin or make an announcement to reach the entire congregation (Slade, 2021). Nevertheless, the finding can be considered good news because most congregations in the US have fewer than 100 members (Chaves & Eagle, 2015), so even small organizations have great potential to reach people with health promotion.

Other churches had lower reach, with the intervention serving fewer than one in three members. This reflects a considerable missed opportunity to benefit members with cancer education. Community Health Advisors at times expressed disappointment and frustration that more people did not come to hear their presentations. Community Health Advisors, after a disappointing initial workshop turnout, often consulted with the study team on active recruitment techniques in an effort to increase participation. As previously discussed, reach was lower in the larger congregations. Mid-size churches that had offered their members nine or more different types of health topics in the past two years had poor reach. Again, this may have resulted in over-saturation of health promotion among the members, making them less inclined to engage, or the Community Health Advisors may have been burnt out and less effective in their recruitment efforts. Finally, churches with 5–10 years of formal health ministry history and that had conducted health promotion activities on an average of 5–8 different health topics in the prior two years had lower reach. This again may be due to over-saturation or competing activities on busy church calendars. Of note, the three factors in the model for higher values of reach (years of having a health ministry in place; membership size; or number of health promotion topics in the past 2 years) were exactly the same factors in the model for lower values of reach. This provides compelling evidence of their difference-making roles, as different values of these three factors corresponded directly to higher and lower values of reach; when these three factors varied, so did the outcome.

Time to Implement the Workshop Series

The Project HEAL intervention was designed with the intent of multiple cancer educational workshops rather than a one-time event and included both: 1) repeated exposure to key cancer risk and early detection information to account for the fact that not every enrollee will attend all workshops; and 2) unique cancer educational information that built upon the previous workshops, to give those who attended multiple workshops new and valuable information each time. Therefore, the timing of the workshops was deliberate, in an attempt to avoid too rapid of a succession that would limit feasibility of attendance yet maintain enough momentum to keep enrollees interested and to facilitate a layered dialogue with the Community Health Advisors based on the material from the previous workshop. Optimally, the three workshops were designed to be delivered approximately monthly, so that all three could be completed in around a 60-day period.

There was variability in the workshop delivery time, with some churches delivering in quicker succession and others taking more time for implementation. It was initially surprising to us that churches that had conducted fewer health promotion activities in the previous two years had more timely implementation. Upon further reflection, however, it is possible that these churches had fewer competing events as opposed to those that were saturated with activities. Furthermore, volunteers assigned by the pastor to implement the intervention may have had fewer commitments, particularly in the area of health activities, and thus were able to move at a quicker pace. Finally, if health promotion were a novel activity for the volunteer interventionists, perhaps enthusiasm translated to a brisker pace.

The combination of a handful of part-time staff and having a pastor who did not have additional employment outside the church likewise linked directly to quicker implementation. This is likely due to the pastor’s ability to focus his or her attention more exclusively on church activities. Finally, the churches with more educated pastors and a mid-size active membership also had more timely implementation. Future research is needed to qualitatively examine how pastor education relates to their support of health ministry activities. It is possible that even if a pastor is highly educated, if he/she heads a small congregation there may be limited resources for implementation, and in larger congregations, there may be many competing activities as well as additional organizational layers (e.g., board/elder approvals, church politics) that can slow implementation. In addition, larger congregations often implement formal procedures to communicate with congregants, which can make it difficult to inform the full congregation about planned health ministry activities (Slade, 2021). Future research should examine ways to optimize reach in large congregations to increase impact of health promotion activities and reduce the resource inefficiency resulting from poor reach.

When considering churches with less timely implementation, similar organizational capacity factors came into play. The two churches where the pastor had outside employment were also those that took more time to implement the workshop series. This stands to reason, as in many churches the pastor is the gatekeeper of decision-making and has multiple employment roles; this can limit the pastor’s availability and slow the approval of workshop scheduling. In addition, in some churches the pastor preferred to attend the workshops to show support, which may have resulted in a lengthier scheduling timeline due to their outside employment obligations.

The combination of a moderate number of health promotion activities in the past two years and a greater number of part-time staff was also related to less timely implementation. Though previous experience and potentially available staff would appear to facilitate implementation, these churches may have also had busier church calendars with many competing activities, resulting in workshop scheduling delays. Collecting data on the church calendar (e.g., number and frequency of events and the scheduling and prioritization process) may have provided helpful insights to understand this finding, as busy church calendars tended to be a key barrier to getting the workshops scheduled (Slade, 2021).

Finally, churches that had both a large number of volunteers and mid-size active congregation size tended to take longer to implement the workshops. Given that the Project HEAL intervention is delivered by volunteers, the number of available volunteers and their portfolio of church commitments comes into play in implementation. This is reinforced by the finding that the number of staff alone did not appear to play a role in implementation timelines. While it may seem counterintuitive that having a large number of volunteers is associated with slower implementation in the mid-size churches, it may be that these congregations offer so many activities that a large volunteer base is needed and could still be insufficient to cover the workload. More nuanced insights are warranted regarding the role of volunteers (e.g., typical number and intensity of activities) and how pastors counted the number of volunteers (e.g., activity threshold for a member being counted as a volunteer). Of note, four of the five factors in the model for higher values of “days to workshop completion” (e.g., number of health activities in the past 2 years, pastor outside employment, number of part-time staff, member weekly attendance) likewise appeared in the model for lower values of “days to completion.” The appearance of these four factors in both models underscores their difference-making roles, as different values of these four factors corresponded directly to high and low values of the outcome; when these four factors varied, so did the outcome.

Organizational Capacity: Is More Better?

A somewhat unexpected finding in our results was that the middle value of an organizational capacity indicator (e.g., member size; number of health activities or topic areas conducted in the previous 2 years), rather than the highest value, was related to better performance on an implementation outcome. An example of this is the reach model, where reach was lower in the larger congregations. Though our previous work evaluated reach and indicated that it was modest overall (Santos et al., 2017), statistical power with church as the unit of analysis limited our previous ability to evaluate reach by church characteristics such as membership size. We previously reported that church member turnout at the workshops was better when the pastor also attended (Williams et al., 2018). Church size and pastor support may work together to influence intervention reach. For example, it may be more difficult for pastors of larger congregations to show their support for church initiatives such as Project HEAL, due to the complexity of the communication structure in these organizations and the high likelihood of concurrent activities.

In the aforementioned analysis of the Faith, Activity, and Nutrition intervention in United Methodist churches (Wilcox et al., 2021), while implementer ratings of inner setting characteristics were associated with implementation outcomes, pastor ratings of these characteristics were not. The study reported that implementation of the healthy eating intervention was greater in churches with fewer than 500 members, a finding consistent with the current finding of better implementation in smaller churches. While the Faith, Activity, and Nutrition study reported that inner setting ratings (culture, readiness) and implementation process (e.g., opinion leader/champion engagement) were most closely linked with implementation of core components, the current study focused more on the structural aspects of capacity (Tagai et al., 2018).

Strengths/Limitations

Study strengths include use of configurational analysis, a cross-case approach that can be applied in studies of differing sample sizes, including small-n studies. Though it would be optimal to have had more intervention sites, this was not feasible given the intensity of the investment required to enroll and then maintain the relationship with each church. In addition, the community-engaged nature of this study overall, from the intervention (Holt et al., 2014) to the instrumentation (Tagai et al., 2018) to the evaluation (Holt et al., 2018; Santos et al., 2017), makes for an authentic and relevant contribution in the area of community-based implementation science. In terms of limitations, there are other indicators across the implementation continuum (e.g., fidelity, sustainment) beyond the scope of the current analysis that should be considered for future study. Factors beyond those included in our original dataset may have played a role in implementation outcomes. Capacity indicators may have served as proxies for other factors not included in the evaluation. In addition, while dichotomizing the outcome variables was necessary for the analysis, doing so can obscure the distribution of the data particularly when scores fall near the median. In the current case, there were a number of churches characterized as having more timely implementation, with number of days to implement the workshop series near sixty. Finally, as previously mentioned, additional qualitative research into the roles of pastors and Community Health Advisors could help explain the configurational findings though within-case and cross-case qualitative analysis that allowed for further examination of these conditions in greater depth and context.

Practice Implications

The current findings can be used to inform future church-based health promotion activities. While the social networks and less complex organizational structure of small churches appears to be conducive to health promotion interventions reaching most members, there are things that larger congregations can do to maximize the benefit of health programming for their members. This may include bolstering communication about health promotion activities both centrally and within all church ministries (e.g., health, outreach, education, family life, women’s, men’s, seniors). This niche saturation approach could help get the word out so that all can benefit. Intentional coordination with the church calendar is advised to avoid situations where multiple events are offered at the same time. While some health promotion activities are one-time events (e.g., health fairs, single educational sessions), those that involve information delivered over multiple sessions should be carefully considered in the context of the recent history of health promotion activities in the church. If there is a precedent for health promotion in the church but not a preponderance of recent health activities, the timing may be right for a more intensive program. In any case, when it comes to health promotion activities offered in the church setting, planning, preparation, and prioritization bode well for success.

Conclusions

This is one of the first studies to apply configurational analysis to evaluate implementation outcomes related to health promotion activities outside of a traditional healthcare setting. A small set of organizational capacity features, primarily membership size and previous health promotion experience, proved to be consistent difference-makers vis-a-vis intervention reach and implementation timeline. Our configurational findings indicate great potential for reaching community members through implementation of evidence-based cancer education in places of worship, particularly in small church congregations, along with identifying the specific aspects of organizational capacity that consistently distinguished churches achieving higher levels of implementation success. Future efforts should focus on scale-up of these activities for population-level impact on cancer disparities.

Availability of data and materials:

Study materials are available upon request from the corresponding author. The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request upon execution of a data use agreement.