Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2024 Sep 1.
Published in final edited form as: Glob Implement Res Appl. 2023 Jun 11;3(3):284–294. doi: 10.1007/s43477-023-00089-0

The role of organizational capacity in intervention efficacy in a church-based cancer education program: A configurational analysis

Cheryl L Knott 1,*, Edward J Miech 2, Nathaniel Woodard 1, Maisha Huq 1
PMCID: PMC10723821  NIHMSID: NIHMS1906781  PMID: 38107832

Abstract

It is well-established in the field of implementation science that the context in which an intervention is delivered can play a crucial role in how well it is implemented. However, less is known about how organizational context or capacity relates to efficacy outcomes, particularly with health promotion interventions delivered outside of healthcare settings. The present study examined whether organizational capacity indicators were linked to key efficacy outcomes in an evidence-based cancer control intervention delivered in 13 African American churches in Maryland. Outcomes included increases in colorectal cancer knowledge and self-report colonoscopy screening behavior from baseline to follow-up. We used Coincidence Analysis to identify features of organizational capacity that uniquely distinguished churches with varying levels of cancer knowledge and screening. Indicators of organizational capacity (e.g., congregation size, prior health promotion experience) were from an existing measure of church organizational capacity for health promotion. A single solution pathway accounted for greater increases in colorectal cancer knowledge over 12 months, a combination of two conditions: conducting 3 or more health promotion activities in the prior 2 years together with not receiving any technical assistance from outside partners in the prior 2 years. A single condition accounted for greater increases in colonoscopy screening over 24 months: churches that had conducted health promotion activities in 1–4 different topical areas in the prior 2 years. Findings highlight aspects of organizational capacity (e.g., prior experience in health promotion) that may facilitate intervention efficacy and can help practitioners identify organizational settings most promising for intervention impact.

Keywords: implementation science, organizational capacity, configurational analysis, coincidence analysis, colorectal cancer, cancer screening

Introduction

A consistent finding within implementation science is that the context or setting in which evidence-based interventions are delivered can impact outcomes across the implementation continuum (Durlak & DuPre, 2008; Gingiss, Roberts-Gray, & Boerm, 2006; Kelly, Baker, Williams, Nanney, & Haire-Joshu, 2004; Norton, 2012; Pfadenhauer et al., 2017; Piat et al., 2021; Riley, Taylor, & Elliott, 2003; Velloza, Roche, Concepcion, & Ortblad, 2023; Woltmann et al., 2008). This has led to the incorporation of organizational factors into a number of contemporary implementation science frameworks, including the Consolidated Framework for Implementation Research (CFIR) (Damschroder et al., 2009), which includes a domain for inner setting (e.g., structure, communication, readiness). The Practice, Robust Implementation and Sustainability Model (PRISM) (Feldstein & Glasgow, 2008) also recognizes that organizational characteristics and infrastructure can impact implementation outcomes. These theories and others (Aarons et al., 2012; Meyers, Durlak, & Wandersman, 2012) have informed a body of implementation science research that examines how local context relates to intervention outcomes.

Organizational capacity is among the factors that fall within the broader domain of local context. Organizational capacity reflects the observable, structural aspects of an organization, and is evaluated at the organizational level (Rabin & Brownson, 2012; Tagai et al., 2018). Examples of organizational capacity include financial, personnel, and material resources. Organizational capacity is related to—but distinct from—organizational readiness (Rabin & Brownson, 2012; Weiner, Amick, & Shoou-Yih, 2008).

A review and meta-analysis of 81 studies concluded that levels of organizational capacity (e.g., staffing and funding) have important implications for the process of implementation (Durlak & DuPre, 2008). Another study reported that financial resources and leadership support were positively associated with intervention adoption in healthcare settings (Norton, 2012), and similar organizational capacity factors were positively associated with implementation fidelity in school-based tobacco prevention programs (Gingiss et al., 2006). Conversely, greater staff turnover was related to lower implementation fidelity and penetration within mental healthcare settings (Woltmann et al., 2008). Staffing has been consistently identified as an important resource for implementation. For example, staffing was positively associated with the number of heart health promotion strategies implemented by public health agencies (Riley et al., 2003), and the number of intervention sessions delivered and subsequent changes in dietary behavior in health education programs (Kelly et al., 2004).

The role of organizational capacity may be particularly evident when health promotion interventions are implemented within limited resource settings. This is likely to be the case when health promotion interventions are implemented in community settings outside of healthcare systems, such as churches, schools, or senior centers. While these settings frequently conduct health promotion activities, the majority of their resources are directed to their primary mission, which is something other than health. This applies to churches, where the primary mission involves faith and ministry, yet these organizations also often serve community support functions including health promotion.

The salience of church organizational capacity to implementation success was recognized by Allen and colleagues (Allen et al., 2015), who designed and implemented an intervention to increase the capacity of Catholic churches with Latino parishioners to provide evidence-based cancer control interventions. The researchers concluded that if interventions can be adapted to fit the church setting, many Catholic churches have the capacity to implement evidence-based cancer control interventions (Allen et al., 2016). A more recent study examined organizational predictors of implementation of the Faith, Activity, and Nutrition intervention in United Methodist churches (Wilcox et al., 2021). The team found that interventionist ratings of inner setting factors such as communication and culture were associated with better implementation of the intervention’s core components. In our own previous research, we reported on organizational capacity factors that were co-present with implementation outcomes from the RE-AIM Framework (Glasgow, Vogt, & Boles, 1999), in the current sample of 13 African American churches from a cancer education intervention delivered in central Maryland (Knott, Miech, et al., 2022). Using configurational analytic methods and coincidence analysis, we found that churches that had better intervention reach were those that had an existing health ministry, or had fewer than 100 members, or mid-size churches (101–499) that had conducted a variety of health promotion activities in the prior 2 years. More timely intervention delivery occurred in churches that had conducted just 1–2 health promotion activities in the prior 2 years or had a small number of part-time staff and a pastor who did not have additional outside employment, or those with a doctorally prepared pastor and a medium size number of weekly attendees (101–249). Findings from this study suggest the importance of organizational capacity in intervention implementation in faith-based community settings (Knott, Miech, et al., 2022). While we previously found evidence of an association between capacity and intervention implementation, the association between organizational capacity and intervention efficacy warrants examination.

The Present Study

Building on our prior work on the role of local context and organizational factors in implementation outcomes (Knott, Miech, et al., 2022), the purpose of this study was to evaluate of the role of organizational capacity factors in key efficacy outcomes in an evidence-based cancer educational intervention delivered in African American churches in Central Maryland. The efficacy outcomes consisted of colorectal cancer knowledge gains from baseline to 12-month follow-up and increases in colonoscopy screening behavior from baseline to 24-month follow-up. To address the research question, we employed Coincidence Analysis, a relatively new member of the broader family known as configurational comparative methods (Whitaker et al., 2020).

The study was rooted in the primary author’s field observation over 20 years of research, that substantial organizational variability exists between churches, and that church-level factors could play a role in intervention outcomes. Churches vary greatly in their resources and capacities and often conduct health promotion activities supporting medically underserved populations. We previously reported that the intervention produced significant pre-post gains in colorectal cancer knowledge as well as self-report receipt of colonoscopy (Knott, C., et al., 2022). However, because the efficacy outcomes varied widely between the churches for this education program, identifying the organizational conditions linked to efficacy outcomes can deepen our understanding of how community-based health-related programs work in practice. This research can in turn be leveraged to maximize the impact of health promotion interventions delivered in limited resource settings.

Methods

Project HEAL Intervention

The intervention platform for this secondary analysis is “Project HEAL” (Health through Early Awareness and Learning) (Holt et al., 2018; Holt et al., 2014; Santos et al., 2017; Santos et al., 2014; Scheirer et al., 2017). In Project HEAL, two lay members (one man and one woman) of African American churches are trained and certified as Community Health Advisors. Trainees complete 13 educational modules and take a cancer knowledge examination. When trainees score 85% or greater on the examination, they become certified to implement a series of three cancer educational workshops (breast, prostate, and colorectal cancer) in their churches. The intervention is designed to increase cancer knowledge and screening behavior. Additional details on the intervention are provided elsewhere (Holt et al., 2014). Project HEAL was implemented in 14 African American churches in Prince George’s County, Maryland. Project HEAL is an evidence-based intervention that showed significant baseline to follow-up increases in breast, prostate, and colorectal cancer knowledge, and significant increases in reports of fecal occult blood test and colonoscopy, mammography maintenance among women, and digital rectal exams in men (Holt et al., 2018).

Measures

Organizational Capacity

Organizational capacity for health promotion was assessed using items from the Faith-Based Organization Capacity Inventory (FBO-CI) (Tagai et al., 2018). This instrument was designed to evaluate organizational capacity specific to faith-based organizations, and theoretically grounded in work by Greenhalgh and colleagues (Greenhalgh, Robert, Macfarlane, Bate, & Kyriakidou, 2004). The FBO-CI includes three subscales: staffing and space (e.g., number of full- and part-time staff; α=.79), health promotion experience (e.g., presence of health ministry; health promotion activities conducted in the past 2 years; α=.83), and external collaboration (e.g., collaborations with outside organizations; participation in research; receipt of technical assistance from outside partners; α=.67) (Tagai et al., 2018). For the current analysis, individual capacity indicators were coded as multi-value ordinal factors with up to 4 levels based on their response scales and distributions (see Table 1).

Table 1.

Organizational Capacity Indicators and Church-level Intervention Outcomes (N=13)

Organizational capacity indicators M SD %
Number of members 241.46 178.87
Pastor not employed outside of the church 84.6
Number of full-time staff 2.38 3.97
Number of part-time staff 3.77 3.40
Number of churchwide volunteers 60.23 37.20
Existing health ministry 69.2
Participants per church 27.54 10.15
Intervention outcomes
Baseline church-level colorectal knowledge score/7 5.57 .39
12-month church-level colorectal knowledge score/7 6.41 .27
Baseline to 12-month change in church-level colorectal cancer knowledge .88 .45
Baseline percentage of participants by church that ever had a colonoscopy 62.99 13.65
24-month percentage of participants by church that ever had a colonoscopy 71.54 12.46
Percentage of participants by church that were not screened at baseline but reported screening at 24 months 9.06 8.16
Open in a new tab

Note. The total number of intervention participants in the study was 358. Church membership size ranged from a minimum of 40 to a maximum of 600 members per church.

Efficacy Outcomes

We identified the two primary intervention efficacy outcomes of colorectal cancer knowledge and colonoscopy screening behavior. While the Project HEAL intervention also had impact on breast and prostate cancer knowledge and screening behavior, the present analysis was limited to the colorectal cancer outcomes, which by including both men and women provided more analytic cases than the sex-specific analyses. Colorectal cancer knowledge was evaluated using an established measure (Green & Kelly, 2004) that included seven items (e.g., “Colorectal cancer screening is not necessary if there are no symptoms”) assessed in agree/disagree/don’t know format with a possible score of 7. Internal reliability was acceptable (0.72) in the study sample (Knott, C., et al., 2022). We examined baseline to 12-month changes in knowledge as there were significant increases which were sustained, but not further increased, by the 24-month follow-up (Knott, C., et al., 2022). A knowledge change score was computed for each church based on the mean pre-post change and then dichotomized for analysis. Based on the data distribution, knowledge gains of .8 points or more represented the positive outcome, and gains of less than .8 represented the negative outcome. This threshold of 0.80 was selected because it corresponded with the difference between the mean baseline church-level colorectal knowledge score 5.57 (out of 7) and the mean 12-month church-level colorectal knowledge score 6.41 (out of 7); this split the larger set of 13 cases into two similarly-sized groups (i.e., 6 churches with the outcome present, 7 with the outcome absent).

To evaluate self-report screening, the Project HEAL trial used established survey items based on the Behavioral Risk Factor Surveillance System (Centers for Disease Control and Prevention, 2003). Participants were asked whether they had ever had a colonoscopy at baseline and again at the 24-month follow-up. The 24-month follow-up interval allowed sufficient time for more participants to come due for screening in accord with guidelines. Based on this survey data, we computed the number of participants in each church that indicated at baseline that they had not had a colonoscopy, but who had indicated at follow-up that they had received the test. We then computed a dichotomous church-level variable for the analysis using a distribution-based threshold in which churches were categorized into those with 10% or more of participants reporting no prior colonoscopy screening at baseline but with screening initiation at follow up (the positive outcome) and those with less than 10% of participants not screened at baseline but screened by the time of follow up (the negative outcome). This threshold of 10% was selected because it corresponded with the difference between the mean baseline percentage of participants by church that ever had a colonoscopy (62.99%) and the mean 24-month percentage of participants by church that ever had a colonoscopy (71.54%); this split the larger set of 13 cases into a group of 5 churches with the outcome present and another 8 with the outcome absent.

Analysis

We used coincidence analysis (CNA) to identify key conditions linked to colorectal cancer knowledge gains and colorectal cancer screening rates. CNA is a mathematical, case-based approach that applies Boolean algebra, logic and set theory to identify the difference-making conditions that uniquely distinguish one group from another. CNA searches for necessary and sufficient conditions for an outcome to appear and models both Boolean conjunctivity (when several conditions must appear together for the outcome to appear) and disjunctivity (when multiple paths lead to an identical outcome) (Furnari et al., 2020; Palinkas, Mendon, & Hamilton, 2019; Ragin, 2014). CNA can be used with samples of various sizes, including small-n studies. CNA has an established and rapidly-growing literature in health, with over 50 peer-reviewed articles appearing in the published literature since 2020 (Cohen et al., 2021; Coury et al., 2021; Hickman, Miech, Stump, Fowler, & Unroe, 2020; Miech et al., 2021; Petrik et al., 2020; Whitaker et al., 2020; Yakovchenko et al., 2020).

Factor Selection

The original dataset had over 15 factors. In the initial exploratory phase of the analysis, we reduced this data using a configurational approach to factor selection described in detail elsewhere (Cohen et al., 2021; Coury et al., 2021; Hickman et al., 2020; Petrik et al., 2020; Yakovchenko et al., 2020). This involved applying the “minimally sufficient conditions” (msc) function within the R “cna” package to exhaustively scan the full dataset in order to identify configurations of conditions with strong connections to the outcomes (e.g., screening rates). This process comprehensively considers all one-, two-, and three-condition configurations present in the dataset, evaluates each configuration against a prespecified threshold for consistency, and then generates a “condition table” that lists and organizes all configurations that satisfy this criterion. In a condition table, rows list configurations of conditions that meet a specified consistency level, with designated columns for outcome, conditions, consistency, and coverage. This exhaustive process considered every combination of values instantiated and identified all 1-, 2-, and 3-condition configurations that met the specified consistency threshold. During this exploratory data analysis, the msc function was run at five different consistency levels: 95%, 90%, 85%, 80%, and 75% (Rich, Miech, Semenza, & Corbin, 2022; Yakovchenko et al., 2020). When reviewing this mathematical output, we focused on configurations with “best of class” coverage scores (i.e., top coverage score among configurations with the same number of conditions) that were also consistent with logic, theory, and prior knowledge. We then used that condition-level information to guide selection of a smaller subset of factors to include in model iteration. We performed this process separately for the two different outcomes: colorectal cancer knowledge gains and colorectal cancer screening rates. Thirteen of the 14 Project HEAL churches had complete data and were included in the analysis.

Model Development

As is standard with configurational analysis, we modeled both higher colorectal cancer knowledge gains (i.e., the presence of the outcome) as well as lower colorectal cancer knowledge gains (i.e., the absence of the outcome), and repeated the process for higher colorectal cancer screening rates and lower colorectal cancer screening rates, yielding a total of four different models. During this phase, we recategorized multi-value factors as dichotomized meta-factors to reduce dimensionality, drawing upon both the mathematical output from the msc routine and the subject-matter knowledge of the project team to establish cut-points for each factor. We then iteratively applied model-building functions within the “cna” software package in R. We assessed models based on their overall consistency and coverage as well as potential model ambiguity (when more than one model satisfies the specified consistency and coverage thresholds and explains the outcome of interest). We selected final models based on all of the following criteria: ≥ 80% scores for both consistency and coverage; the same set of factors (taking on different values) explained both the presence and the absence of the outcome, and no model ambiguity (Adams, Miech, & Sobieraj, 2022; Davis et al., 2022; Rich et al., 2022). The Coincidence Analysis package (“cna”) in R (Ambühl, Baumgartner, Epple, Parkkinen, & Thiem, 2021), R (version 3.5.0), and R Studio (version 1.1.383) were used to support the analyses.

Results

Table 1 shows the organizational capacity indicators, intervention outcomes, and their distributions. Over half of the churches in the sample were in the medium size range of 101–499 members. The churches had considerably more volunteers than part- or full-time paid staff. Roughly three-fourths of the churches reported having an existing health ministry. Baseline to 12-month knowledge about colorectal cancer increased across the churches almost 1 point out of a possible 7 points. Baseline within-church reports of participants having ever had a colonoscopy ranged from 44.4% to 84.6%. Among participants who provided data at both baseline and follow-up, the percentage of participants by church who were not screened at baseline but reported screening at 24 months ranged from 0% to 22.2%. These data reflected considerable between-church heterogeneity in the outcomes.

CNA Analyses: Colorectal Cancer Knowledge

Table 2 shows the CNA model for greater increases in colorectal cancer knowledge. This model had one solution pathway, a conjunct of two conditions related to organizational capacity: churches that conducted more health promotion activities in the prior 2 years (3 or more activities) and had not received any technical assistance from outside partners in the prior 2 years. CNA models have two main parameters of fit: consistency and coverage. Consistency indicates how reliably a model yields an outcome, and is calculated by the number of cases identified by a model that also have the outcome of interest divided by “the total number of cases identified by the model.” This solution for higher colorectal cancer knowledge gains had 86% consistency (6/7), as there were six cases identified by the model that also had the outcome (n=6) divided by seven cases total that were identified by the model (n=7). Coverage refers to the explanatory breadth of a model and is calculated by the number of cases identified by a model that also have the outcome of interest divided by the total number of cases that have the outcome of interest. This solution had 100% coverage (6/6), as there were six cases identified by the model that also had the outcome (n=6) divided by six cases total that had the outcome of interest (n=6).

Table 2.

Configurational Model for Greater Increases in Colorectal Cancer Knowledge (N=13)

Church ID Colorectal Cancer

Knowledge Increases		 Level of Recent

Health Activities		 # Partners Providing

Technical Assistance
3 Larger 2 a 0 a
6 Larger 3 a 0 a
10 Larger 2 a 0 a
11 Larger 2 a 0 a
12 Larger 2 a 0 a
19 Larger 3 a 0 a
4 Smaller 3 1
8 Smaller 3 1
14 Smaller 1 0
15 Smaller 3 1
16 Smaller 1 0
17 Smaller 2 b 0 b
18 Smaller 1 0
Open in a new tab

Note. The bolded dashed line divides cases with the outcome of interest present (above the line) from those without (below the line). In the third column, 0=0 health promotion activities in prior two years, 1=1–2 activities, 2=3–4 activities, and 3=5 or more activities in the prior two years.

a

indicates cases covered by the model that also have the outcome present.

b

indicates inconsistent cases, cases identified by the model but did not have the outcome present.

Table 3 shows the configurational model for smaller gains in colorectal cancer knowledge. This model had two distinct solution pathways composed of just one condition each: churches that had conducted few (1–2) health promotion activities in the prior 2 years OR that had received technical assistance from 1 or more outside partners in the prior 2 years. Each solution pathway was sufficient by itself for the outcome of interest. This configurational model for smaller gains had both 100% consistency (6/6) and 100% coverage (6/6). The same two factors appear in both the positive and negative models, underscoring their roles as difference-makers, as the outcome varied when these two factors took on different values.

Table 3.

Configurational Model for Smaller Increases in Colorectal Cancer Knowledge (N=13)

Church ID Colorectal Cancer

Knowledge Increases		 Level of Recent

Health Activities		 # Partners Providing

Technical Assistance
4 Smaller 3 1 a
8 Smaller 3 1 a
14 Smaller 1 a 0
15 Smaller 3 1 a
16 Smaller 1 a 0
17 Smaller 2 0
18 Smaller 1 a 0
3 Larger 2 0
6 Larger 3 0
10 Larger 2 0
11 Larger 2 0
12 Larger 2 0
19 Larger 3 0
Open in a new tab

Note. The bolded dashed line divides cases with the outcome of interest present (above the line) from those without (below the line). In the third column, 0=0 health promotion activities in prior two years, 1=1–2 activities, 2=3–4 activities, and 3=5 or more activities in the prior two years.

a

indicates cases covered by the model that also have the outcome present.

CNA Analyses: Colorectal Cancer Screening

Table 4 shows the configurational model for greater increases in colonoscopy. This model had one pathway consisting of a single sufficient condition: churches that had conducted health promotion activities in 1–4 different topical areas in the prior two years. This model had 80% consistency (4/5) and 80% coverage (4/5).

Table 4.

Configurational Model for Greater Increases in Colonoscopy Utilization (N=13)

Church ID Increase in Colonoscopy Utilization Breadth of Health

Promotion Topics
10 Larger 1 a
14 Larger 1 a
16 Larger 1 a
17 Larger 2
18 Larger 1 a
3 Smaller 3
4 Smaller 3
6 Smaller 3
8 Smaller 3
11 Smaller 2
12 Smaller 1 b
15 Smaller 2
19 Smaller 3
Open in a new tab

Note. The bolded dashed line divides cases with the outcome of interest present (above the line) from those without (below the line). In the third column, 0=health promotion in 0 areas in the prior 2 years, 1=1–4 areas, 2=5–8 areas, and 3=health promotion in 9 or more areas in the prior 2 years.

a

indicates cases covered by the model that also have the outcome present.

b

indicates inconsistent cases, cases identified by the model but did not have the outcome present.

Table 5 shows the configurational model for smaller increases in colonoscopy. This model also had a single pathway, consisting of a lone sufficient condition: churches that had conducted health promotion activities in 5 or more different topical areas in the prior 2 years. This configurational model had 88% consistency (7/8) and 88% coverage (7/8). Again, the same factor appeared in both the positive and negative models, underscoring its role as a difference-maker.

Table 5.

Configurational Model for Smaller Increases in Colonoscopy Utilization (N=13)

Church ID Increase in Colonoscopy Utilization Breadth of Health

Promotion Topics
6 Smaller 3 a
3 Smaller 3 a
4 Smaller 3 a
8 Smaller 3 a
11 Smaller 2 a
12 Smaller 1
15 Smaller 2 a
19 Smaller 3 a
10 Larger 1
14 Larger 1
16 Larger 1
17 Larger 2 b
18 Larger 1
Open in a new tab

Note. The bolded dashed line divides cases with the outcome of interest present (above the line) from those without (below the line). In the third column, 0=health promotion in 0 areas in the prior 2 years, 1=1–4 areas, 2=5–8 areas, and 3=health promotion in 9 or more areas in the prior 2 years.

a

indicates cases covered by the model that also have the outcome present.

b

indicates inconsistent cases, cases identified by the model but did not have the outcome present.

Discussion

The current study identified aspects of organizational capacity that can help explain key efficacy outcomes in a trial that evaluated an evidence-based colorectal cancer control intervention delivered by volunteers in African American churches. While prior research on organizational capacity for health promotion in church settings indicated that these settings do have the capacity to implement evidence-based cancer control interventions (Allen et al., 2016), the current findings begin to elucidate which aspects of capacity play a role in their efficacy. Another prior church-based study indicated that inner setting factors such as communication and culture were associated with better implementation (Wilcox et al., 2021), whereas the current study focused more on structural aspects of capacity.

We previously identified organizational factors that were present along with desirable implementation outcomes of reach and implementation time frame (Knott, Miech, et al., 2022), and cautiously anticipated that greater organizational capacity would also be related to better intervention efficacy outcomes. In the present analysis, there were 3 church organizational capacity indicators that were consistently related to the efficacy outcomes of colorectal cancer knowledge and colonoscopy screening. These were recent health promotion experience (both the number of activities and breadth of health topics) and receipt of outside technical assistance to support the church’s operations. We begin by discussing the role that these factors played in the intervention efficacy outcomes.

Colorectal Cancer Knowledge

The Project HEAL intervention was designed to increase knowledge about colorectal cancer among participants who attended the series of three educational workshops taught by the two trained and certified Community Health Advisors in each participating church. Churches where the workshop participants had greater colorectal cancer knowledge gains over the study period were those that were active in conducting health promotion activities in the prior two years and reported having not received technical assistance from outside partners in that same time period. This combination of conditions reflects churches that were experienced in conducting health promotion activities and were self-sufficient to the degree that they did not engage outside partners to help meet their programming needs. Conversely, churches in which the workshop participants reported fewer gains in colorectal knowledge were those that had done very little health promotion in the prior 2 years or those that had received outside technical assistance. Neither of these indicators were sufficient on their own for smaller knowledge gains. Our previous study reported a “Goldilocks” finding in which intervention reach was greater in small to mid-size churches relative to larger churches, and churches reporting some, but not extensive health promotion experience (Knott, Miech, et al., 2022). In contrast, the current findings suggest a more linear relationship between organizational capacity and the knowledge efficacy outcome. Finally, while church size played a role in the previously reported implementation outcomes (where smaller churches fared better), it did not come into play with the efficacy outcomes, further reinforcing the idea that one needs to look beyond congregation size to understand these organizations in the context of health promotion.

Colonoscopy Screening

A primary goal of the Project HEAL intervention is to encourage age-eligible people to engage in colorectal cancer screening behavior. Only one of our organizational capacity indicators, number of different health promotion topics covered by the churches in the 2 years prior to the intervention, was related to colonoscopy screening among members. Churches in which more people reported initiation of screening over the study period were those that had conducted health promotion activities in a small number of topics over the prior 2 years, while churches that covered a greater breadth of topics in that period had fewer people reporting screening. While seemingly counterintuitive, it is possible that people in churches that were very active in health promotion had a number of competing health priorities on their minds due to the greater exposure to various health messaging topics, which could have decreased their likelihood of getting a colonoscopy.

Conversely, churches where the members had received less variety of health messaging in this setting were those that had the better screening outcomes. It could be that the novelty of the Project HEAL content was powerful enough to stimulate screening behavior in this relatively intervention-naïve population. This idea is consistent with our previous finding that intervention reach was better in mid-size churches that had conducted health promotion activities on fewer health topics.

Organizational Capacity Indicators as Difference-Makers

While our previous analysis of organizational capacity and implementation outcomes identified 6 indicators that served as difference-makers to differentiate churches with desirable and less desirable implementation outcomes (Knott, Miech, et al., 2022), only three factors related to organizational capacity were difference-makers for the outcomes of colorectal cancer knowledge gains and screening rates. This may reflect a proximal-distal distinction, where a greater number of organizational factors influence the more immediate outcomes related to intervention implementation than to the more distant outcomes related to intervention efficacy. It is notable that two of the three indicators that appeared in the models in the current analysis were also present in the implementation analysis: prior recent health promotion experience in the form of frequency of health activities and breadth of health topics addressed.

It is also possible that the relationships between organizational factors and implementation and efficacy outcomes depend on the nature of the intervention or other host setting factors. In the context of a schoolwide positive behavior intervention other researchers reported better implementation and student-level efficacy outcomes in the smaller schools (Pas & Bradshaw, 2012). Likewise, a large CNA study of over 200 public elementary and secondary schools in Oregon found that the conditions linked to higher or lower student absenteeism rates varied by a school’s grade level (Coates, 2021). Conversely, church size was not related to the current efficacy outcomes. The variation in findings related to organizational size and intervention efficacy could be explained by differences in the nature of the intervention (e.g., level, intensity), the intended intervention outcomes, or the settings themselves.

Though several studies have identified staffing as important for intervention implementation (Durlak & DuPre, 2008; Woltmann et al., 2008; Riley et al., 2003; Kelly et al., 2004), staffing was not a significant driver of efficacy outcomes in the current analysis. Perhaps this is a function of differences between the types of organizations being studied. For example, in a healthcare system, staff are responsible for implementation whereas in the current study the churches generally had few paid staff and the intervention was delivered by volunteers. Similarly, while previous research highlighted the role of financial resources in implementation outcomes (Durlak & DuPre, 2008; Norton, 2012), this aspect of capacity was not linked with efficacy in the current study. This could be in part because the current intervention was provided free to the churches.

Strengths/Limitations

Study strengths include an emphasis on health disparities and African American populations that are disproportionately burdened by cancer (American Cancer Society, 2019). This study prioritized church organizations serving predominantly Black populations. Historical and continued maltreatment of Black communities in society and in healthcare systems and institutions have contributed to medical mistrust and low utilization of care in traditional healthcare settings among these populations (Powell et al., 2019; Randolph et al., 2020; Rogers & Bryant-Davis, 2020; Wells & Gowda, 2020). Given this, the current study aimed to reach people where they are rather than through a healthcare or other “outside” system. With the historical role of Black churches as trusted organizations that support communities of color, the current work has direct implications for addressing health disparities via intervention in faith-based settings. In particular, the current findings point to aspects of organizational capacity (e.g., prior health promotion activities) that can help identify churches where health promotion interventions are likely to have the greatest impact.

The use of CNA analysis, a cross-case approach that can be applied even in small-n studies is another strength of the current study. Though it would have been desirable to have had more intervention sites, this was not feasible given the intensity of the investment required to enroll and then maintain the relationship with each church. We were able to examine a variety of indicators of organizational capacity from leadership to partnerships to experience, to identify those that were co-present with desirable intervention outcomes.

In terms of limitations, these same organizational capacity indicators may have served as proxies for other factors not included in the evaluation. It would be beneficial to conduct qualitative inquiry to further understand the findings, for example to glean insights as to what motivated the intervention participants to get screened. A recent study in seven community mental health organizations found a non-linear pattern between organizational capacity and implementation outcomes, leading the authors to call for qualitative data as imperative for interpreting the findings (Piat et al., 2021). Qualitative inquiry may help elucidate the mechanisms of the relationships observed in the current study and whether the indicators of organizational capacity involved in those relationships could be a proxy for additional factors to examine in future research. With respect to screening, the study had to rely on self-report from the study participants, as medical record validation of screening is not feasible given the number of healthcare systems where people in the region access screening. Finally, we characterize the indicators of organization capacity as Boolean difference-makers that could be viewed as “possible causes” or “candidate causes,” pending additional confirmation via independent replication in other studies: we cannot ascertain the direction and magnitude of causal relationships from the present analysis. It is possible that other factors (e.g., participant-level factors) not included in the dataset could have contributed to between-church variation in screening behavior.

Conclusions

The current study identified indicators of organizational capacity directly connected to efficacy outcomes of a health promotion intervention that aimed to increase cancer knowledge and screening behavior, delivered in faith-based settings. Future research and practice within these settings may consider beginning with an organizational capacity assessment to better know the strengths and limitations of host organizations. In cases where knowledge is the primary aim of the intervention and baseline awareness may be limited as in the case of colorectal cancer, particular attention may be paid to the church’s recent history of health promotion activities and the extent to which they engage outside technical assistance to support their overall programming. When behavior change is the goal, a careful look at the content of prior health promotion activities may be warranted. The current findings add to the growing evidence base that intervention setting plays an important role in the outcomes of health initiatives. Data that can shed light on the types of organization where an intervention is likely to be most effective can help policymakers and public health practitioners direct resources to the settings where they are likely to have the greatest impact.

Acknowledgements:

The team would like to thank Col. Jimmie Slade and Ms. Barbara Jean Shaneman-Robinson, who conducted the community engagement activities for the present study.

Funding:

This work was supported by the National Cancer Institute under Grant R01CA147313 and by funds through the Maryland Department of Health’s Cigarette Restitution Fund Program.

Footnotes

Declarations

Ethics approval and consent to participate: All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. All study methods were approved by the University of Maryland Institutional Review Board (#10–0691), and all participants provided written informed consent to participate.

Consent for publication: All authors have provided consent for publication.

Availability of data and materials: Study materials are available upon request from the corresponding author.

Clinical trial registration: The study was pre-registered at clinicaltrials.gov. Identifier: NCT02076958. Registered 3/4/2014. https://clinicaltrials.gov/ct2/show/NCT02076958

Competing and financial interests: The authors declare that they have no competing or financial interests.

References

  1. Aarons GA, Green AE, Palinkas LA, Self-Brown S, Whitaker DJ, Lutzker JR, . . . Chaffin MJ (2012). Dynamic adaptation process to implement an evidence-based child maltreatment intervention. Implementation Science, 7, 32. doi: 10.1186/1748-5908-7-32 [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Adams KK, Miech EJ, & Sobieraj DM (2022). Factors that distinguish opioid withdrawal during induction with buprenorphine microdosing: a configurational analysis. Addiction Science & Clinical Practice, 17(1), 55. doi: 10.1186/s13722-022-00336-z [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Allen JD, Torres MI, Tom LS, Leyva B, Galeas AV, & Ospino H (2016). Dissemination of evidence-based cancer control interventions among Catholic faith-based organizations: results from the CRUZA randomized trial. Implementation Science, 11, 74. doi: 10.1186/s13012-016-0430-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Allen JD, Torres MI, Tom LS, Rustan S, Leyva B, Negron R, . . . Ospino H. (2015). Enhancing organizational capacity to provide cancer control programs among Latino churches: design and baseline findings of the CRUZA Study. BMC Health Services Research, 15, 147. doi: 10.1186/s12913-015-0735-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Ambühl M, Baumgartner M, Epple R, Parkkinen V-P, & Thiem A (2021). Cna: causal modeling with coincidence analysis Retrieved from https://cran.microsoft.com/snapshot/2021-10-10/web/packages/cna/cna.pdf
  6. American Cancer Society. (2019). Cancer facts & figures for African Americans 2019–2021 Retrieved from Atlanta, GA: https://www.cancer.org/content/dam/cancer-org/research/cancer-facts-and-statistics/cancer-facts-and-figures-for-african-americans/cancer-facts-and-figures-for-african-americans-2019-2021.pdf
  7. Centers for Disease Control and Prevention. (2003). Behavioral risk factor surveillance system: U.S. Department of Health and Human Services
  8. Coates KE (2021). Affecting absenteeism through school-based health services delivery: A configurational comparative methods study of Oregon’s public secondary schools. (Ph.D.). Portland State University, Oregon. Retrieved from https://www.proquest.com/docview/2572591242 ProQuest Dissertations & Theses Global database (28546306) [Google Scholar]
  9. Cohen DJ, Sweeney SM, Miller WL, Hall JD, Miech EJ, Springer RJ, . . . Marino M (2021). Improving smoking and blood pressure outcomes: The interplay between operational changes and local context. Annals of Family Medicine, 19(3), 240–248. doi: 10.1370/afm.2668 [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Coury J, Miech EJ, Styer P, Petrik AF, Coates KE, Green BB, . . . Coronado GD (2021). What’s the “secret sauce”? How implementation variation affects the success of colorectal cancer screening outreach. Implementation Science Communications, 2, 5. doi: 10.1186/s43058-020-00104-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, & Lowery JC (2009). Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implementation Science, 4, 50. doi: 10.1186/1748-5908-4-50 [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Davis MM, Schneider JL, Petrik AF, Miech EJ, Younger B, Escaron AL, . . . Coronado GD (2022). Clinic Factors Associated With Mailed Fecal Immunochemical Test (FIT) Completion: The Difference-Making Role of Support Staff. The Annals of Family Medicine, 20(2), 123–129. doi: 10.1370/afm.2772 [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Durlak JA, & DuPre EP (2008). Implementation matters: A review of research on the influence of implementation on program outcomes and the factors affecting implementation. American Journal of Community Psychology, 41(3–4), 327–350. doi: 10.1007/s10464-008-9165-0 [DOI] [PubMed] [Google Scholar]
  14. Feldstein AC, & Glasgow RE (2008). A practical, robust implementation and sustainability model (PRISM) for integrating research findings into practice. The Joint Commission Journal on Quality and Patient Safety, 34(4), 228–243. doi: 10.1016/s1553-7250(08)34030-6 [DOI] [PubMed] [Google Scholar]
  15. Furnari S, Crilly D, Misangyi VF, Greckhamer T, Fiss PC, & Aguilera R (2020). Capturing causal complexity: Heuristics for configurational theorizing. Academy of Management Review, 46(4), 778–799. doi: 10.5465/amr.2019.0298 [DOI] [Google Scholar]
  16. Gingiss PM, Roberts-Gray C, & Boerm M (2006). Bridge-It: A system for predicting implementation fidelity for school-based tobacco prevention programs. Prevention Science, 7(2), 197–207. doi: 10.1007/s11121-006-0038-1 [DOI] [PubMed] [Google Scholar]
  17. Glasgow RE, Vogt TM, & Boles SM (1999). Evaluating the public health impact of health promotion interventions: The RE-AIM framework. American Journal of Public Health, 89(9), 1322–1327. doi: 10.2105/ajph.89.9.1322 [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Green PM, & Kelly BA (2004). Colorectal cancer knowledge, perceptions, and behaviors in African Americans. Cancer Nursing, 27(3), 206–215. doi: 10.1097/00002820-200405000-00004 [DOI] [PubMed] [Google Scholar]
  19. Greenhalgh T, Robert G, Macfarlane F, Bate P, & Kyriakidou O (2004). Diffusion of innovations in service organizations: Systematic review and recommendations. The Milbank Quarterly, 82(4), 581–629. doi: 10.1111/j.0887-378X.2004.00325.x [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Hickman SE, Miech EJ, Stump TE, Fowler NR, & Unroe KT (2020). Identifying the implementation conditions associated with positive outcomes in a successful nursing facility demonstration project. Gerontologist, 60(8), 1566–1574. doi: 10.1093/geront/gnaa041 [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Holt CL, Tagai EK, Santos SLZ, Scheirer MA, Bowie J, Haider M, & Slade JL (2018). Web-based versus in-person methods for training lay community health advisors to implement health promotion workshops: Participant outcomes from a cluster-randomized trial. Translational Behavioral Medicine, 9(4), 573–582. doi: 10.1093/tbm/iby065 [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Holt CL, Tagai EK, Scheirer MA, Santos SL, Bowie J, Haider M, . . . Whitehead T (2014). Translating evidence-based interventions for implementation: Experiences from Project HEAL in African American churches. Implementation Science, 9, 66. doi: 10.1186/1748-5908-9-66 [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Kelly CM, Baker EA, Williams D, Nanney MS, & Haire-Joshu D (2004). Organizational capacity’s effects on the delivery and outcomes of health education programs. Journal of Public Health Management and Practice, 10(2), 164–170. doi: 10.1097/00124784-200403000-00011 [DOI] [PubMed] [Google Scholar]
  24. Knott CL, C., C., Bowie J, Mullins CD, Slade J, Woodard N, . . . He X (2022). Cluster randomized trial comparing organizationally-tailored vs. standard approach for integrating an evidence-based cancer control intervention into African American churches. Translational Behavioral Medicine, 12(5), 673–682. doi: 10.1093/tbm/ibab088 [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Knott CL, Miech EJ, Slade J, Woodard N, Robinson-Shaneman B-J, & Huq M (2022). Evaluation of organizational capacity in the implementation of a church-based cancer education program. Global Implementation Research and Applications, 2(105), 1–11. doi: 10.1007/s43477-021-00033-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Meyers DC, Durlak JA, & Wandersman A (2012). The quality implementation framework: a synthesis of critical steps in the implementation process. American Journal of Community Psychology, 50(3–4), 462–480. doi: 10.1007/s10464-012-9522-x [DOI] [PubMed] [Google Scholar]
  27. Miech EJ, Freitag MB, Evans RR, Burns JA, Wiitala WL, Annis A, . . . Damschroder LJ (2021). Facility-level conditions leading to higher reach: a configurational analysis of national VA weight management programming. BMC Health Services Research, 21, 797. doi: 10.1186/s12913-021-06774-w [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Norton WE (2012). An exploratory study to examine intentions to adopt an evidence-based HIV linkage-to-care intervention among state health department AIDS directors in the United States. Implementation Science, 7, 27. doi: 10.1186/1748-5908-7-27 [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Palinkas LA, Mendon SJ, & Hamilton AB (2019). Innovations in mixed methods evaluations. Annual Review of Public Health, 40, 423–442. doi: 10.1146/annurev-publhealth-040218-044215 [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Pas ET, & Bradshaw CP (2012). Examining the association between implementation and outcomes: State-wide scale-up of school-wide positive behavior intervention and supports. The Journal of Behavioral Health Services and Research, 39(4), 417–433. doi: 10.1007/s11414-012-9290-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Petrik AF, Green B, Schneider J, Miech EJ, Coury J, Retecki S, & Coronado GD (2020). Factors influencing implementation of a colorectal cancer screening improvement program in community health centers: An applied use of configurational comparative methods. Journal of General Internal Medicine, 35(Suppl 2), 815–822. doi: 10.1007/s11606-020-06186-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Pfadenhauer LM, Gerhardus A, Mozygemba K, Lysdahl KB, Booth A, Hofmann B, . . . Brereton L (2017). Making sense of complexity in context and implementation: the Context and Implementation of Complex Interventions (CICI) framework. Implementation science, 12(1), 1–17. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Piat M, Wainwright M, Cherkas D, Leblanc S, Sofouli E, Rivest M-P, . . . Labonté L (2021). Identifying and understanding the contextual factors that shaped mid-implementation outcomes during the COVID-19 pandemic in organizations implementing mental health recovery innovations into services. Implementation Science Communications, 2, 1–16. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Powell W, Richmond J, Mohottige D, Yen I, Joslyn A, & Corbie-Smith G (2019). Medical mistrust, racism, and delays in preventive health screening among African-American men. Behavioral Medicine, 45(2), 102–117. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Rabin BA, & Brownson RC (2012). Developing the terminology for dissemination and implementation research. In Brownson RC, Colditz GA, & Proctor EK (Eds.), Dissemination and implementation research in health: Translating science to practice (pp. 23–54). New York, NY: Oxford University Press. [Google Scholar]
  36. Ragin CC (2014). The comparative method: Moving beyond qualitative and quantitative strategies Oakland, CA: University of California Press. [Google Scholar]
  37. Randolph SD, Golin C, Welgus H, Lightfoot AF, Harding CJ, & Riggins LF (2020). How Perceived Structural Racism and Discrimination and Medical Mistrust in the Health System Influences Participation in HIV Health Services for Black Women Living in the United States South: A Qualitative, Descriptive Study. The Journal of the Association of Nurses in AIDS Care : JANAC, 31(5), 598–605. doi: 10.1097/JNC.0000000000000189 [DOI] [PubMed] [Google Scholar]
  38. Rich JA, Miech EJ, Semenza DC, & Corbin TJ (2022). How combinations of state firearm laws link to low firearm suicide and homicide rates: A configurational analysis. Preventive Medicine, 165, 107262. doi: 10.1016/j.ypmed.2022.107262 [DOI] [Google Scholar]
  39. Riley BL, Taylor SM, & Elliott SJ (2003). Organizational capacity and implementation change: a comparative case study of heart health promotion in Ontario public health agencies. Health Education Research, 18(6), 754–769. doi: 10.1093/her/cyf051 [DOI] [PubMed] [Google Scholar]
  40. Rogers G, & Bryant-Davis T (2020). Historical and contemporary racial trauma among Black Americans: Black wellness matters. In Geffner R, White JW, Hamberger LK, Rosenbaum A, Vaughan-Eden V, & Vieth VI (Eds.), Handbook of Interpersonal Violence and Abuse Across the Lifespan: A project of the National Partnership to End Interpersonal Violence Across the Lifespan (NPEIV) (pp. 1–35). Cham: Springer International Publishing. [Google Scholar]
  41. Santos SLZ, Tagai EK, Scheirer MA, Bowie J, Haider M, Slade J, . . . Holt CL (2017). Adoption, reach, and implementation of a cancer education intervention in African American churches. Implementation Science, 12, 36. doi: 10.1186/s13012-017-0566-z [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Santos SLZ, Tagai EK, Wang MQ, Scheirer MA, Slade JL, & Holt CL (2014). Feasibility of a web-based training system for peer community health advisors in cancer early detection among African Americans. American Journal of Public Health, 104(12), 2282–2289. doi: 10.2105/ajph.2014.302237 [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Scheirer MA, Santos SL, Tagai EK, Bowie J, Slade J, Carter R, & Holt CL (2017). Dimensions of sustainability for a health communication intervention in African American churches: A multi-methods study. Implementation Science, 12, 43. doi: 10.1186/s13012-017-0576-x [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Tagai EK, Holt CL, Scheirer MA, Santos SLZ, Haider M, Bowie J, . . . Wang MQ (2018). Assessing capacity of faith-based organizations for health promotion activities. Health Promotion Practice, 19(5), 414–723. doi: 10.1177/1524839917737510 [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Velloza J, Roche S, Concepcion T, & Ortblad KF (2023). Advancing considerations of context in the evaluation and implementation of evidence-based biomedical HIV prevention interventions: a review of recent research. Current Opinion in HIV and AIDS, 18(1), 1–11. [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Weiner BJ, Amick J, & Shoou-Yih DL (2008). Review: Conceptualization and measurement of organizational readiness for change. Medical Care Research and Review, 65(4), 379–436. doi: 10.1177/1077558708317802 [DOI] [PubMed] [Google Scholar]
  47. Wells L, & Gowda A (2020). A legacy of mistrust: African Americans and the US healthcare system. Proceedings of UCLA health, 24, 1–3. [Google Scholar]
  48. Whitaker RG, Sperber N, Baumgartner M, Thiem A, Cragun D, Damschroder L, . . . Birken S (2020). Coincidence analysis: a new method for causal inference in implementation science. Implementation Science, 15, 108. doi: 10.1186/s13012-020-01070-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Wilcox S, Jake-Schoffman DE, Saunders RP, Kinnard D, Kaczynski AT, Hutto B, & James KL (2021). Predictors of implementation in the Faith, Activity, and Nutrition dissemination and implementation study: application of the Consolidated Framework for Implementation Research (CFIR) in a statewide initiative. Translational Behavioral Medicine, 11(2), 419–429. doi: 10.1093/tbm/ibaa025 [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. Woltmann EM, Whitley R, McHugo GJ, Brunette M, Torrey WC, Coots L, . . . Drake RE (2008). The role of staff turnover in the implementation of evidence-based practices in mental health care. Psychiatric Services, 59(7), 732–737. doi: 10.1176/ps.2008.59.7.732 [DOI] [PubMed] [Google Scholar]
  51. Yakovchenko V, Miech EJ, Chinman MJ, Chartier M, Gonzalez R, Kirchner JE, . . . Rogal SS (2020). Strategy configurations directly linked to higher hepatitis c virus treatment starts: an applied use of configurational comparative methods. Medical Care, 58(5), e31–e38. doi: 10.1097/mlr.0000000000001319 [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES