Advancing Health Equity through Implementation Science: Identifying and Examining Measures of the Outer Setting

Abstract

Background

Implementation science (IS) could accelerate progress toward achieving health equity goals. However, the lack of attention to the outer setting where interventions are implemented limits applicability and generalizability of findings to different populations, settings, and time periods. We developed a data resource to assess outer setting across seven centers funded by the National Cancer Institute’s IS Centers in Cancer Control (ISC³) Network Program.

Objective

To describe the development of the Outer Setting Data Resource and characterize the county-level outer context across Centers.

Methods

Our Data Resource captures seven key environments, including: (1) food; (2) physical; (3) economic; (4) social; (5) health care; (6) cancer behavioral and screening; and (7) cancer-related policy. Data were obtained from public sources including the US Census and American Community Survey. We present medians and interquartile ranges based on the distribution of all counties in the US, all ISC³ centers, and within each Center for twelve selected measures. Distributions of each factor are compared with the national estimate using single sample sign tests.

Results

ISC³ centers’ catchment areas include 458 counties and over 126 million people across 28 states. The median percentage of population living within ½ mile of a park is higher in ISC³ counties (38.0%, interquartile range (IQR): 16.0%-59.0%) compared to nationally (18.0%, IQR: 7.0%-38.0%; p<0.0001). The median percentage of households with no broadband access is significantly lower in ISC³ counties (28.4%, IQR: 21.4%-35.6%) compared the nation overall (32.8%, IQR: 25.8%-41.2%; p<0.0001). The median unemployment rate was significantly higher in ISC³ counties (5.2%, IQR: 4.1%-6.4%) compared to nationally (4.9%, 3.6%-6.3%, p=0.0006).

Conclusions

Our results indicate that the outer setting varies across Centers and often differs from the national level. These findings demonstrate the importance of assessing the contextual environment in which interventions are implemented and suggest potential implications for intervention generalizability and scalability.

Introduction

In the past two decades, the importance of increasing efforts to implement evidence-based programs in ways that ensure that disadvantaged groups have access to evidence-based practices and policies has been increasingly recognized. Regardless of setting, implementation efforts occur within an ecology that will both impact and be impacted by the implementation approach. One of the many contributions of implementation science (IS), defined as the use of scientific methods to find the best ways to integrate proven, effective interventions into routine health care (Bauer et al., 2015), is the specification of aspects of context that may impact implementation efforts. IS has the potential to accelerate progress toward achieving health equity goals, and significant work has focused on implementation in settings that serve historically disadvantaged populations (Baumann & Cabassa, 2020; Brownson et al., 2021). By accounting for the influence of the outer setting, or the area-level characteristics of the implementation setting, we are better equipped to explain how or why certain implementation outcomes are achieved, and to ensure generalizability to a wider range of settings (Nilsen & Bernhardsson, 2019). Knowledge about context is also critical if we are to understand variations in outcomes (Edwards & Barker, 2014; May et al., 2016; Nilsen & Bernhardsson, 2019; Tomoaia-Cotisel et al., 2013).

There are many implementation theories that include aspects of context, highlighting the importance of considering setting-level factors. Nilsen describes the characteristics of context that are included in 8 determinant frameworks, all largely focused on organizational factors (Nilsen & Bernhardsson, 2019). For example, the Consolidated Framework for Implementation Research (CFIR) defines the context of the “inner setting” of an organization as having five key elements (Damschroder et al., 2009). These include the organization’s structural characteristics (e.g. social structure, maturity, size), formal and informal communications and networks, culture (e.g. the organizations’ norms and values), implementation climate (e.g. capacity for change and shared receptivity to an intervention, perceived need for change, fit of the intervention with the setting, perceived priority and incentives associated with the intervention, goal alignment, and learning climate), and readiness for implementation (e.g. leadership engagement, resource availability, access to knowledge about how to incorporate the intervention into work tasks).

Some of the determinant frameworks, particularly CFIR, the Exploration, Preparation, Implementation, and Sustainment Framework (EPIS), PARIHS (Promoting Action on Research Implementation in Health Services) also consider the outer setting--that is the factors that are external to the organization that might influence implementation of an evidence-based intervention. EPIS, for example, considers system-level leadership, policies, regulations, and legislation in the service environment, the financing of care across municipal, state, and federal levels, and inter-organizational networks (Aarons et al., 2011; Braveman et al., 2011; Moullin et al., 2019). CFIR defines the outer setting as comprised of patient needs and resources, inter-organizational networks, peer and/or competitive pressure, and external policies, incentives, and mandates (Damschroder et al., 2009). The original PARIHS framework narrowly defined context as characteristics of the organizational setting (Kitson et al., 2008), however, the revision, i-PARIHS, includes external context (Harvey & Kitson, 2016). In fact the creators acknowledge that lack of consideration for the external social, political, policy and economic context in which implementation occurs was a key reason for revising the framework. Still, outer setting is frequently poorly defined or goes unexamined and/or unreported in implementation studies (Ashcraft et al., 2020; Brownson et al., 2021).

Another aspect of the outer setting that is less well-articulated in these theories relates to the social determinants that influence both the organization, its staff, and its clients. The World Health Organization defines social determinants of health as “conditions in which people are born, grow, live, work, and age, and the wider set of forces and systems shaping the conditions of daily life” (World Health Organization, 2022). As we plan and implement interventions for the prevention, early detection, and treatment of cancer, we argue that it is increasingly important to take this aspect of the outer setting into account, especially as we consider the entrenched nature of inequities in cancer prevention and control.

Social determinants of health flow from structural factors that are key influencers of health inequality and contribute to health disparities between populations and communities. To date, little to no attention has been given to these structural aspects of the outer setting in IS, though other work has shown the importance of structural and neighborhood factors as determinants of cancer prevention and care (Gomez et al., 2015; Namin et al., 2021). It is naïve to think that implementing organizations are not impacted by the structure of their community context. It is in that context that employees are hired and work, patients live, and organizations must acquire goods and services to support their business and health care mission. Failure to fully measure the structural elements of outer setting and to use that data to inform implementation efforts limits the applicability, adaptability, and generalizability of study findings to different populations, settings, and time periods (Brownson et al., 2021; Nilsen & Bernhardsson, 2019). Brownson et al. (Brownson et al., 2021) rightly argue that to advance equity we need to more fully account for macro-level historical, cultural, economic, and political forces that shape implementation in low-resource settings and communities. Consideration of the outer setting is an important step in this direction.

There is considerable scientific evidence documenting that where a person lives predicts health, and that the wider set of forces and systems shaping the conditions of daily life can influence health outcomes more than medical care does (Heiman HJ, 2015; Marmot et al., 2008). Cancer epidemiology research identifies patterns of social disadvantage (i.e. social determinants of health), such as adverse working and living conditions, and are associated with the adoption of unhealthy lifestyle behaviors and poor health outcomes related to cancer prevention and control (Carethers & Doubeni, 2020; Coughlin, 2019; Qin et al., 2021). Notably, a recent study showed the importance of considering many neighborhood level indicators over just a few or combined indices (Kolak et al., 2020). They showed that neighborhood indicators accounted for 71% of the variance across census track in the US, reinforcing the geographic heterogeneity of the US population. It therefore stands to reason that these same factors may impact the ability of a health system or community organization to effectively implement evidence based interventions (EBIs) designed to improve equity, as the staff and system itself are likely operating within the shadow of those same factors. This is because the outer setting may influence both implementation outcomes such as acceptability, adoption, appropriateness and sustainability as well as clinical or behavioral outcomes of the intervention such as efficiency, effectiveness, or equity (Proctor et al., 2011) through access to and availability of resources, policies which facilitate or create barriers to change, or unanticipated events local, regional, or global events that cause disruption disrupt implementation and/or delivery. A failure to include these structural, community, and health system determinants in the development and testing of implementation strategies will limit population health goals for achieving equitable implementation of evidence-based interventions (EBIs) across populations and settings.

This paper describes an effort undertaken by 7 centers funded by the National Cancer Institute’s (NCI) Implementation Science Centers in Cancer Control (ISC³) Network Program. The Network, participating centers, and their funded projects are described in detail in Oh et al., 2023. In brief, ISC³ supports the development, testing, and refinement of innovative approaches to implement evidence-based cancer control interventions, using unique implementation partnerships ranging from oncology or primary care to community services at each of the 7 centers (Oh et al., 2023). The ISC³ Network represents virtually all US regions including the: Northeast (Massachusetts), Mid-Atlantic (Pennsylvania), Midwest (Missouri and Illinois), South (North Carolina), and West (Colorado, Oregon, Washington) (Figure 1). The network further represents a wide range of implementation settings (urban primary care, rural primary care, specialty care), populations (e.g., adult primary care patients, cancer patients, cancer survivors), and points on the cancer continuum (e.g., primary prevention, screening and early detection, and cancer treatment and survivorship). Through a supplement mechanism from the NCI, we created a robust set of common data elements that describe the outer setting of each Center’s implementation setting. We have also collected data needed to characterize this context for each Center, and thus characterize the outer implementation context across the seven ISC³ centers. To our knowledge this work represents the first effort to conduct an environmental scan that will characterize the structural outer implementation context in a comprehensive manner and that represents such a diverse range of settings across the US.

Figure 1.

United States Counties by ISC³ Center

Collecting these data will allow us to investigate the extent to which this is true, how this varies across differing geographies and target populations of participating Centers, and determine which specific contextual measures matter most. This will facilitate a better understanding of how implementation approaches can be tailored to address local area contexts. The goal of this paper is to describe creation of the Outer Contextual Data Resource, conduct an environmental scan of the outer setting in which our implementation centers are operating, and discuss potential future applications.

Methods

The ISC³ Network and Outer Context Working Group

Each of the seven ISC³ Network Centers selected a representative to join the Outer Context Working Group, with a goal to conduct an environmental scan of social determinants of health within site-specific geographic areas. The working group was responsible for selecting the common data elements, collecting the data, building a cross-center data repository, conducting analyses, and disseminating results. Detail on each Center is provided in Table 1.

Table 1.

Description of ISC³ Centers

ISC3 Center	Implementation Theme	Implementation Laboratory Composition by organizational type	Counties in Catchment Area
Building Research in Implementation and Dissemination to close Gaps and achieve Equity in Cancer Control Center (BRIDGE-C2) at Oregon Health & Science University	Advancing implementation science to improve cancer screening and prevention in underserved populations.	OCHIN Network: 391 Community Health Centers; 45 Health Departments; 69 Safety net+; 20 Non-profit+; 290 Other	Number of counties: 299* Total population: 114,524,090 Metro counties: 65.6%
Colorado Implementation Science Center in Cancer Control at University of Colorado (Colorado ISC3)	Using pragmatic approaches to assess and enhance the value of cancer prevention and control in rural primary care; and interactive resources to enhance implementation science capacity.	4 Network/Health System+	Number of counties: 47 Total population: 710,041 Metro counties: 0%
Implementation and Informatics – Developing Adaptable Processes and Technologies for Cancer Control (iDAPT) at IDAPT School of Medicine/University of Massachusetts Medical School	Using technologies to support rapid cycle and real-time deployment and testing of implementation processes and adaptations within cancer control.	3 Network/Health System+ 1 Hospital	Number of counties: 14 Total population: 2,170,996 Metro counties: 78.6%
The Implementation Science Center for Cancer Control Equity (ISCCCE) at the ISCCCE T.H. Chan School of Public Health	Improving community health by integrating health equity and implementation science for evidence-based cancer control.	32 Community Health Centers, 1 primary care association	Number of counties: 9 Total population: 6,486,544 Metro counties: 100%
Optimizing Implementation in Cancer Control (OPTICC) at University of Washington and Kaiser Permanente Washington Health Research Institute14,15	Developing, testing, refining, and disseminating innovative methods for optimizing the implementation of evidence-based interventions in cancer control.	6 Network/Health System+; 1 Health Department	Number of counties: 29 Total population: 7,133,275 Metro counties: 65.5%
Penn Implementation Science Center in Cancer Control (Penn ISC3) at University of Pennsylvania	Applying insights from behavioral economics to rapidly accelerate the pace at which evidence-based practices for cancer care are deployed and to which they are delivered equitably, thereby increasing their reach and impact on the health of individuals with cancer.	6 Network/Health System+; 6 Hospitals	Number of counties: 7 Total population: 4,638,023 Metro counties: 100%
Washington University Implementation Science Center for Cancer Control (WU-ISCCC) at Washington University in St. Louis	Building a rigorous, scientific evidence base for rapid-cycle implementation research to increase the reach, external validity, and sustainability of effective cancer control interventions.	5 Network/Health System+; 3 Community health center; 11 Patient Advisory Group/Community Member; 2 Health Department; 7 Advocacy; 3 Academic+; 3 Health Association; 2 Non-profit+; 2 Other+	Number of counties: 82 Total population: 4,817,436 Metro counties: 31.7%

^*29 of 299 counties overlap with another Center

Rationale and Selection of Contextual Factors to Define the Outer Setting

Measures of interest represent seven domains to capture the following environments: (1) food; (2) physical; (3) economic; (4) social; (5) health care; (6) cancer behavioral and screening; and (7) cancer-related polices (Table 2; a full list is available in Supplementary Table 1). While there is some conceptual overlap between domains, our focus was to identify a comprehensive set of key measures, more so than to definitively delineate which domain was the best fit. These measures, which draw heavily from Community Vital Signs (Bazemore et al., 2016) and the Social Determinants of Health Index (Wang et al., 2021), were selected to efficiently characterize the outer setting in which cancer prevention and control programs are provided. The selected factors capture elements of These domains and measures of interest capture multiple dimensions of the outer setting in which ISC³ interventions are implemented and have been associated with diverse health outcomes including diabetes, mental health, and cancer mortality (Bilal et al., 2018; Pinheiro et al., 2022; Sui et al., 2022).

Table 2.

Domains and Selected Social Structural Outer Context Common Data Elements (SC-CDE) of Interest

Domain	Example Measures	County-level measure included in this analysis
Food	Grocery Store Access % on SNAP Modified Retail Food Environment Index Supermarkets and Other Grocery (except Convenience) Stores (total and per 100,000 population)	Percent of people living more than 1 mile from a supermarket or large grocery store if in an urban area, or more than 10 miles from a supermarket or large grocery store if in a rural area. Obtained from the 2015 USDA Food Atlas.
Physical	Access to parks and recreation spaces Urban classification code Housing (e.g., modernity, ownership) Crime Pollution Infrastructure	Percentage of population living within half a mile of a park. Obtained from CDC’s National Environmental Public Health Tracking Network, 2015. Percentage of households with no broadband obtained from American Community Survey (ACS), 2016-2020.
Economic	Poverty and employment Residential housing (e.g., foreclosures, mortgages, vacancies, evictions, costs, lending) Commercial real estate (e.g., vacancies, density)	Percentage of population unemployed. Obtained from ACS, 2016-2020. Percentage of population below 200% of Federal Poverty Level. Obtained from ACS, 2016-2020. Social deprivation index, a composite measure of area level deprivation derived from seven demographic characteristics collected in the American Community Survey. Higher scores indicate greater deprivation. Index obtained from the Robert Graham Center, 2019.
Social	Population characteristics (e.g., density, racial/ethnic composition, segregation) Education (e.g., % with high school diploma) Language (e.g., % with limited English proficiency) Transportation (e.g., % with access to a car)	Percent of households with no vehicle. Obtained from ACS 2015-2019 Median monthly housing costs. Obtained from ACS 2016-2020
Health status and Healthcare	% in poor physical or mental health % with disability Access to care (e.g., primary care provider density, health insurance)	Percentage of adults with no usual source of care. Obtained from BRFSS, 2017
Cancer health behaviors and screening	Proportion of population with cancer risk factors (e.g., smoking, overweight/obesity, fruit & veg intake, physical activity, alcohol intake) % Not current with cancer screening, overall and by type Extent of disparities in cancer screening, overall and by type % Completed HPV vaccination Extent of disparities in HPV vaccination completion	Average percentage of Medicare enrollees age 67-69 having at least one mammogram over a two-year period, 2019. Obtained from Dartmouth Atlas. Age-adjusted prevalence of fecal occult blood test, sigmoidoscopy, or colonoscopy among adults aged 50-75 years, 2018. Obtained from CDC PLACES.
Cancer-related policies	Medicaid expansion Medicaid coverage of comprehensive tobacco treatment Local and state tobacco policies State SNAP incentive programs for fruit and vegetable access	Smokefree Requirements for bars, restaurants, and workplaces. Obtained from NCI GIS Portal for Cancer Research Website Tobacco Policy.

We identified measures of the food environment that include measures of the retail food environment, grocery store and other food store access including alcohol outlets, and food insecurity. Access to supermarkets have been associated with greater consumption of fruits and vegetables, lower obesity rates, and better diet quality (Centers for Disease Control and Prevention, 2014). Areas with few supermarkets often have a higher proportion of convenience stores with limited stocks of fresh produce, and fast food restuarants that each offer low-cost calorically dense foods (Althoff et al., 2022; Bower et al., 2014). Alcohol consumption, a risk factor for multiple cancers (Cao & Giovannucci, 2016), also varies by neighborhood availability with greater concentration and advertisement in disproportionately lower income and minority communities (Scott et al., 2020). Food insecurity, in which households or individuals have insufficient resources to secure adequate access to food, is associated with poor physical and metal health, including depression, diabetes and poor sleep (Gundersen & Ziliak, 2015).

For the physical environment domain, we selected a number of measures related to the built environment such as walkability and access to green spaces which affect physical activity and sedentary behavior (Hermelink et al., 2022; McTiernan et al., 2019); air and water pollution which can increase exposure to carcinogens (C. Lin et al., 2015a; Turner et al., 2020); and physical infrastructure like broadband access that may indirectly impact health behaviors and access to care. Walkability has been associated with increased moderate and vigorous physical activity, and decreased sedentary time, among both working-age and older adults across varying levels of income (King et al., 2011; Sallis et al., 2009; Watson et al., 2020). Access to green spaces, including parks and other natural facilities, as well as recreational facilities such as has also been associated with increased moderate and vigorous physical activity and decreased sedentary time in a wide variety of settings and populations (Bedimo-Rung et al., 2005; Cerin et al., 2008; O’donoghue et al., 2016). Air pollution is thought to contribute to the development of cancer via the absorption, metabolism, and distribution of inhaled carcinogens (Turner et al., 2020). Air pollution is considered a Group 1 carcinogen for lung cancer, with more limited evidence for other forms of cancer, including bladder and breast cancer. Water pollution, particularly arsenic, nitrates, and chromium, have also been associated with increased risk of skin, kidney, and bladder cancer (L. Lin et al., 2022).

Selected area-level economic measures selected include measures of income and income inequality, poverty, unemployment, use of public assistance, residential housing measures such as foreclosures, evictions, and housing burden costs. Area-level measures of household income and proportion of population living under the federal poverty level have been widely studied with demonstrated associations with cancer incidence, treatment, and survival (Akinyemiju et al., 2016; Hastert et al., 2015; Snider et al., 2023). Greater area-level unemployment was associated with lower levels of smoking abstinence among Black participants in a smoking cessation randomized clinical trial (Kendzor et al., 2012), while greater economic inequality contributes to diffusion of new clinical interventions as demonstrated by the lower uptake of breast cancer clinical genomic testing in areas with high income inequality (Ponce et al., 2015). Greater municipal-level rental costs area associated with greater maternal morbidity (Muchomba et al., 2022). Living in census tracts with high-foreclosure-risk was associated with lower self-rated health among women with breast cancer (Schootman et al., 2012) and weight gain among community-dwelling adults (Arcaya et al., 2013).

Constructs of interest within the social environment included education, childhood development, population demographics, transportation, residential segregation, and redlining. Measures of area-level educational attainment are associated with cancer stage at diagnosis, with individuals living in census-tracts with low proportions of adults with a high school diploma having 12% higher odds of having advanced-stage lung cancer at the time of diagnosis (Gupta et al., 2022). Access to public transit, car ownership and other transportation measures may influence realized access to healthcare, resources, and public goods. Greater access to public transit is associated with greater physical and mental health and can increase access to healthcare and healthy foods (Heaps et al., 2021). County-level racial and socioeconomic residential segregation, as measured by the Index of Concentration at the Extremes, is associated with cancer mortality. Individuals living in areas with greater residential racial and economic segregation as measured by the Index of Concentration at the Extremes (ICE), had increased mortality from 12 of 13 examined cancer sites, with greatest risks among individuals with lung and bronchus cancer (Zhang et al., 2023). People living in areas with greater degrees of historic redlining have higher risk of late stage cancer and greater cancer mortality (Collin et al., 2021; Krieger et al., 2020) while racial bias in mortgage lending was associated with worse colorectal cancer survival among Black women in southeastern Wisconsin (Zhou et al., 2017).

Within health and health care we attempted to identify measures that capture access to care, health insurance and health care costs, and population measures of physical and mental health. Area-level access to health care is associated with healthcare utilization (Allen et al., 2017), cancer prevalence (Rereddy et al., 2015), receipt of adjuvant chemotherapy for colorectal cancer (C. C. Lin et al., 2015b), and the concentration of low access areas in locations with greater populations of racial and ethnic minority and lower socioeconomic status contributes to health disparities (Snowden & Michaels, 2023; Streeter et al., 2020). Greater county-level physician density per 100,000 population is associated with lower rates of late-stage colorectal cancer diagnoses (Ananthakrishnan et al., 2010).

Constructs examined within the cancer behaviors and screening and cancer-related policies domains included cancer screening rates, smoking prevalence, insurance coverage for cancer screening, skin tanning, and tobacco control. Tobacco control measures vary across geographic areas within the US (Buettner-Schmidt et al., 2019). State and local cigarette taxes, prohibitions on indoor smoking, and minimum age to legally purchase cigarettes affect individual’s exposure to secondhand smoke (Perlman et al., 2016), can influence smoking initiation, and can affect ability, willingness, and success with quitting smoking (Wilson et al., 2012). There is also some evidence that state and local tobacco control policies can influence participation in smoking cessation treatment (Thrul et al., 2021).

Data Sources

Data sources include multiple publicly available datasets, including the US Census, American Community Survey, US Department of Agriculture, US Department of Housing and Urban Development, the Centers for Disease Control and Prevention, and the Environmental Protection Agency and other data sources. The specific sources of selected measures are available in Table 1 and Supplementary Table 1.

Data Collection and Database Development

Each ISC³ site contracted with HealthLandscape, a subsidiary of the American Academy of Family Physician’s Graham Center, which maintains an extensive database of curated data elements, derived measures, and calculated indices. HealthLandscape provides data at the level of state, county, zip code tabulation area, and census tract for the entire US, as available, updated on a quarterly basis. Any data elements selected by the working group that were not available from HealthLandscape were either requested or collected from the original source by the working group. For example, state and local policies related to cancer control were identified and collected separately.

The data repository was built in REDCap, a HIPAA-compliant data capture platform, and hosted by one Center, the Implementation Science Center for Cancer Control Equity (ISCCCE) at the Harvard T.H. Chan School of Public Health, and accessible to analysts from each Center. The database utilizes the REDCap API and REDCap R packages in the R statistical programming environment (R Core Team, 2021) to access and modify data in REDCap via Application Programming Interface (Beasley, 2020; Nutter & Lane, 2020). This study was approved by the Mass General Brigham Human Research Committee (Protocol # 2021P003486) and by the respective institutional review boards of each participating center.

Statistical Analysis

Each center provided a list of county-level Federal Information Processing System (FIPS) codes corresponding with their catchment area. Several centers had overlapping counties. These FIPS codes were used to link with county-level data for each selected SC-CDE. 2013 Rural Urban Continuum Codes (United States Department of Agriculture Economic Research Service, 2013) were used to determine whether each ISC³ county was metro (codes 1-4) or non-metro (codes 5-9). Measures presented in this analysis (Table 2) were selected from the broader repository using the following criteria: 1) at least one measure per domain; 2) data available at county level; 3) relatively easy to interpret and understand how it might affect or relate to implementation settings. Medians and interquartile ranges are presented by Center, based on the counties included in each Center’s catchment area, as well as for the entire US. The single sample sign test was used to produce p-values for the comparison between each centers’ median and the national median for the different selected measures. All statistical analyses were conducted in R (R Foundation for Statistical Computing, Vienna, Austria).

Results

Figure 1 displays the geographic distribution of ISC³ counties across the United States. The 458 counties included in ISC³ centers’ catchment areas cover a wide geographic range including 28 states and most US regions. ISC³ counties include over 126 million people as of the 2016-2020 American Community Survey and 52.6% are considered metro counties based on 2013 Rural Urban Continuum Codes.(USDA United States Department of Agriculture Economic Research Service, 2013)

Physical Context:

Figures 2a-c graph the median and interquartile range for each selected outer setting measure across all ISC³ counties, across all US counties, and within counties included in each Center’s catchment area. Figure 2a illustrates that, with respect to the food environment, the median proportion of individuals in a county living more than 1 miles from a supermarket or large grocery store in an urban (or >10 miles for rural areas) is similar in all ISC³ counties (20.0%, IQR: 12.2%-27.4%) as compared to the US as whole (19.1%, IQR: 10.8%-28.8%; p=0.21). County-level medians in individual centers range from 17.9% (IQR: 11.0%-21.7%) in iDAPT counties to 29.0% (IQR: 24.8%-31.7%) in ISCCCE counties. The counties represented by two Centers (OPTICC and ISCCCE) had significantly higher percentages of the population living more than 1 mile (or 10 miles for rural areas) from a supermarket than the national median. None were significantly lower. With respect to access to green spaces, the median percentage of population living within ½ mile of a park is significantly higher in ISC³ counties (38.0%, IQR: 16.0%-59.0%) compared to the national median (18.0%, IQR: 7.0%-38.0%; p<0.0001) (Figure 2b). The range for counties included in ISC³ Centers extends from a median of population with nearby park access of 11.0% (IQR: 7.5%-16.8%) in iDAPT counties to 61.0% (IQR: 48.0%-70.0%) in ISCCCE counties; no Centers had a significantly smaller proportion of their counties’ populations living near a park, compared with the national median, while four (Penn ISC3, BRIDGE-C2, ISCCCE, and Colorado ISC) had significantly higher proportions. Access to broadband is another measure of physical infrastructure. The median percentage of households with no broadband access is significantly lower in ISC³ counties (28.4%, IQR: 21.4%-35.6%) compared to the nation overall (32.8%, IQR: 25.8%-41.2%; p<0.0001) (Figure 2c). Center medians range from to 18.5% (IQR: 14.9%-20.7%) in ISCCCE counties to 36.0% (IQR: 29.9%-41.3%) of the county-level population with no broadband access in WU-ISCCC counties; one Center (WU-ISCCC) has significantly lower levels of broadband access among its counties’ populations, compared to national levels, while five Centers (OPTICC, iDAPT, ISCCCE, BRIDGE-C2, Penn ISC3) had significantly higher levels of broadband access compared to the national median.

Figure 2.

County-level distribution of selected outer context data elements across ISC³ centers

A. Percentage of people in an urban (rural) county living more than 1 (10) mile(s) from a supermarket or large supermarket

B. Percentage of population living within one half mile of a park, 2015

C. Percentage of households with no broadband, 2016-2020

D. Percentage of adults unemployed, 2016-2020

E. Percentage of population below 200% of the federal poverty level, 2016-2020

F. Social Deprivation Index

G. Percentage of households with no vehicle, 2015-2019

H. Median monthly housing costs, 2016-2020

I. Percent of adults with no usual source of care, 2017

J. Average percentage of female of Medicare enrollees aged 67 – 69 having at least one mammogram over a two-year period, 2019

K. Age-adjusted prevalence of fecal occult blood test, sigmoidoscopy, or colonoscopy among adults aged 50-75 years, 2018

L. Presence of Smokefree Laws (Bar, Restaurant, and/or Workplace) in ISC3 Counties

Economic Context:

Figures 2d-2f display the county-level distribution of selected economic measures across ISC³ Centers. The median unemployment rate was significantly higher in ISC³ counties (5.2%, IQR: 4.1%-6.4%) compared to the national level (4.9%, 3.6%-6.3%, p=0.0006) (Figure 2d). Among Centers, median county-level unemployment rates ranged from 4.3% (IQR: 3.7%-5.7%) in Colorado ISC3 to 5.6% (IQR: 5.1%-6.4%) in OPTICC. Unemployment rates were significantly higher than the national median in two Centers (BRIDGE-C2 and OPTICC), while none were significantly lower. The proportion of individuals with household income less than 200% of the federal poverty level was modestly but significantly less prevalent in ISC³ counties (32.5%, IQR: 26.0%-38.1%) compared to the US (34.2%, IQR: 27.7%-41.0%; p=0.001) (Figure 2e). The median percentages of population with household income less than 200% of the federal poverty level across the Centers’ counties range from 22.1% (IQR: 16.7%-25.0%) in ISCCCE to 35.6% (IQR: 33.2%-38.7%) in iDAPT counties. Three Centers (OPTICC, ISCCCE, and BRIDGE-C2) had percentages that were significantly lower than the national median while none were significantly greater. Median social deprivation index scores (higher scores indicate greater deprivation) were similar in ISC³ counties (43.5, IQR: 22.0-67.0) compared to the nation (42.0, IQR: 18.0-66.0; p=0.26) (Figure 2f). Across Centers’ counties, median social deprivation index scores ranged from 26.0 (IQR: 14.0-56.0) in Colorado ISC3 to 50.0 (IQR: 40.0-55.8) in iDAPT counties. One Center (Colorado ISC3) had lower social deprivation scores than the national median, while none were greater.

Social Context:

Figures 2g-2i display the county-level distribution of social and health status/healthcare measures across ISC³ Centers. The percent of households with no vehicle was similar across all ISC³ counties (5.8%, IQR: 4.4%-7.4%) and nationally (5.6%, IQR: 4.2%-7.5%; p=0.24). The medians across Centers ranged from 5.0% (IQR:4.4% – 6.0%) in Colorado ISC3 to 10.5% (IQR: 9.4%-10.6%) in ISCCCE counties (Figure 2g), and medians for two Centers (ISCCCE and WU-ISCCC) and were significantly above the national level while none were significantly lower. Median monthly housing costs are significantly higher in ISC³ counties ($869.50, IQR: $696.75-$1148.75) compared to the national level ($717.00, IQR: $601.75-$898.00; p=<0.0001) (Figure 2h). Median monthly housing costs across Centers ranged from $662.00 (IQR: $600.50-$749.50) in WU-ISCCC to $1618.00 (IQR: $1354.00-$1765.00) in ISCCCE counties and were significantly above the national median in two Centers (ISCCCE and Penn ISC3) and significantly lower in one (WU-ISCCC).

Healthcare/Health Policy:

The percentage of adults with no usual source of medical care was slightly but significantly higher in ISC³ counties (20.9%, IQR: 17.9%-24.0%) compared to national levels (20.4%, IQR: 17.2%-24.2%; p=0.01) (Figure 2i). The prevalence of no usual source of care across Centers at the county level ranged from 11.6% (IQR: 10.2%-11.7%) in ISCCCE to 24.1% (IQR: 21.5%-27.2%) in Colorado ISC3 counties; three Centers (Colorado ISC3, OPTICC, and BRIDGE-C2), and had county-level medians that were significantly higher than the national level while three (ISCCCE, WU-ISCCC, and Penn ISC3) were significantly lower. County-level measures of cancer health behaviors and screening and cancer-related policies are displayed in Figures 2j-2l. The median percentage of Medicare enrollees aged 67-69 having at least one mammogram in the past two years was similar in ISC³ counties (64.7%, IQR: 58.7%-69.4%) and nationally (63.9%, IQR: 57.8%-69.2%; p=0.35) (Figure 2j). County-level medians across Centers ranged from 55.1% (IQR: 50.0%-59.9%) in Colorado ISC3 to 76.8% (IQR: 76.5%-77.3%) in ISCCCE. Proportions with a mammogram were significantly lower in one Center (Colorado ISC3) and significantly higher in three Centers (iDAPT, ISCCCE, and BRIDGE-C2). Age-adjusted prevalence of colorectal cancer screening (stool-based tests, sigmoidoscopy, or colonoscopy) among adults aged 50-75 years was slightly but significantly higher in ISC³ counties (63.7%, IQR: 61.3%-66.3%) than the US overall (62.1%, IQR: 58.9%-65.2%; p <0.0001) (Figure 2k). Median county-level colorectal cancer screening rates across Centers ranged from 60.5% (IQR: 57.9%-62.6%) in Colorado ISC3 to 69.5% (IQR: 68.6%-71.2%) in ISCCCE, with the median at one Center (Colorado ISC3) being significantly below the national level and three (iDAPT, ISCCCE, and BRIDGE-C2) significantly higher. Only three of seven ISC³ centers (Colorado ISC3, BRIDGE-C2, and ISCCCE) have 100% smokefree laws (prohibiting smoking in bars, restaurants, and workplaces) in place in all counties in their catchment area. The remaining centers include counties with a mix of laws covering bars, restaurants, or workplaces, but not all three.

Discussion

The data presented highlight the variations in several contextual indicators across and within Centers. The findings also show that for most indicators except proportion of population lving below 200% of the federal poverty level and the social deprivation index, at least one center’s catchment area significantly differs from the national average. Given this variation, it is important to consider the contextual environment of the settings in which interventions are implemented, but also the implications for the generalizability and scalability of these interventions. Generalizability and scalability are key elements in implementation science (Trinkley et al., 2022) and are critical to expanding the reach of evidence-based interventions and advancing health equity. Generalizabilty is often evaluated based on the characteristics of the inner setting, but here we argue that the outer setting should be considered as well. The findings from this environmental scan demonstrate the variability in contextual factors that define the outer setting and given the known association of these factors with cancer and other health-related outcomes, suggest that these differences may have implications for intervention implementation and dissemination. We showed that some centers have significantly different averages than national estimates, some with narrow and other wide variations across these different indicators. Interventions developed and implemented in catchment areas with small variations in specific contextual indicators (e.g., targeting broadband access to facilitate telemedicine) may not be as successful in those with greater variations (Wilcock et al., 2019). Therefore, in addition to describing the inner setting of an organization, identifying outer contextual factors related to the intervention goals and or implementation strategies will be important to determine the adaptations needed for successful scalability in other settings and geographic areas.

We posit that there are three key ways in which outer setting data could be used in IS including: 1) support selection of EBIs for implementation; 2) design tailored implementation strategies; and 3) evaluate the effectiveness of implementation strategies across settings. These approaches are not mutually exclusive and outer and inner setting data could be examined together to comprehensively understand the influence of setting on implementation outcomes. There may be several available EBIs designed to address any given health problem, incorporating outer setting into the selection process would mean considering whether the evidence demonstrating intervention effectiveness is generalizable across populations and settings. For example, selection of an intervention aimed at increasing physical activity among cancer survivors should differ (e.g., virtual at home exercise vs. park-based group exercise classes) if implementation occurs in an area with greater versus limited access to green spaces such as parks. Development or selection of context appropriate EBIs should be done in concert with community partners and is key to advancing health equity in IS (Brownson et al., 2021; Kerkhoff et al., 2022). However, data on how outer setting affects implementation of one EBI vs. another is currently limited by the dearth of literature that addresses these factors when reporting implementation outcomes.

Selection of implementation strategies may also be influenced by outer contextual factors. In another example, we might consider two outer contextual factors, distance to supermarkets and car ownership together. For example if one wished to implement an educational intervention to increase fruit and vegetable intake among patients in several community health centers distributed across multiple geographic areas, you might use outer setting data to determine how food access varies in the health centers’ catchment areas and use that to tailor the selection of community partners and the accompanying implementation strategies. Partnerships with local farmers markets or food trucks might be more beneficial in areas with low car ownership and greater distance to supermarkets. In this case, food and transportation access data could be used for targeted recruitment of organizations to ensure the intervention matches the setting’s structural needs. Lastly, you may use outer contextual data to evaluate the effectiveness of implementation strategies. For example, following implementation of an EBI designed to reengage patients in primary care where one of the implementation strategies was to provide technical support for telemedicine, you may wish to examine how area-level measures of access to broadband and wireless internet affected the success of this strategy. As described earlier, the more we do this as a standard part of implementation evaluation, the easier it will be to select EBIs and tailor implementation strategies.

For this environmental scan, we focused on county as the geographic unit as it was the smallest unit available for all the data elements. Smaller units such as zip codes or census track would undoubtedly show more variation across and within centers and provide greater precision in understanding the outer settings and its clients (Chen & Krieger, 2021). Future work using outer contextual factors should carefully consider which levels best fit the indicators that are most likely to impact intervention implementation. Importantly, the outer setting must also be understood in the context of the state or county policies. For instance, a specific catchment area may have lower than average rates of uninsurance because their state implemented a state-wide health insurance coverage program, but higher than average costs of living. In this state, intervention implementation would likely be more successful if it considered financial barriers over access barriers. In other states, the structural context would be different and thus require attention to different barriers. Consequently, the different configurations of social structural issues will likely have a differential impact on the different implementation targets.

In conclusion, variations in living and working conditions may affect the ability of interventions to be successfully implemented and disseminated in different settings without adaptations, especially for EBIs that were not developed or tested in similar settings. Fortunately, advances in geospatial technology and public access to a wealth of contextual information enable implementation scientists to evaluate and characterize their intervention setting and promote a greater understanding of which strategies or adaptations to strategies are needed to ensure uptake of evidence-based practices. We’ve created a dynamic repository that will be updated as new data are identified or derived. Specific analyses will be added to the repository and documentation and will be available for use in approved collaborative studies. ISC³ Centers can choose to do their analyses from the larger database using all or selected Centers, or to create a separate database for Center-specific analyses that integrate local data. We believe this flexible structure will facilitate the examination and incorporation of contextual data into implementation strategy development and evaluation. We recommend characterizing the contextual characteristics of the catchment area of the targeted setting to better design, evaluate, and disseminate interventions.

Highlights.

1. Insufficient attention is paid to the outer setting in which interventions are implemented

2. This limits the applicability and generalizability of findings, and health equity advances

3. There is significant variation in the outer setting across implementation sites

4. Evaluating the outer setting is key to ensure that interventions meet community needs