Organizational readiness to implement bundled interventions to increase HIV linkage and retention in care and treatment: results from the Black Women First (BWF) initiative

Abstract

Background

Evidence-based and evidence-informed interventions designed to address gaps in the HIV care continuum have the potential to improve HIV care and treatment. However, inadequate organizational readiness can derail intervention uptake, prevent the integration of interventions, and contribute to suboptimal HIV treatment outcomes. This study sought to understand organizational readiness to implement bundled interventions for Black women with HIV and inform facilitators and barriers to implementation.

Methods

We conducted a mixed methods readiness assessment across 12 sites participating in the Black Women First (BWF) initiative to gauge preparedness to implement bundled interventions. Readiness was assessed using the organizational readiness for implementing change (ORIC) scale, and two open-ended questions examined facilitators and barriers. Associations between participant and organizational level factors were evaluated using linear models with clustering by site at baseline, 6- and 12-months. Pre-implementation interviews were conducted with staff virtually and transcripts were managed in NVivo. Directed content analysis was used to explore implementation barriers and facilitators.

Findings

Sites demonstrated high levels of organizational readiness at baseline; overall organizational readiness for implementing change (ORIC) (mean 56.4, median 59, interquartile range [IQR] 5) and subscales of the ORIC change efficacy (mean 32.4, median 35, IQR 4), change commitment (mean 24, median 25, IQR 1), which is consistent with willingness and capability to implement bundled interventions for Black women with HIV. Organizational readiness remained high at 6- and 12-month follow-up periods. Staff role was significantly associated with organizational readiness (p = 0.007), change efficacy (p = 0.006), and change commitment (p = 0.020) at 6 months. Qualitative analysis indicated strategic planning and assessment (e.g., team coordination and the development of workflows to support implementation); organizational change through network weaving across silos within the organization, and communications systems that engage external partners, as well as resources available for hiring and training, supported readiness. Collaborative leadership and organizational buy-in, staff motivation, and partnerships facilitated implementation processes.

Conclusions

Organizations in the BWF initiative have high levels of organizational readiness reflecting willingness and capability to implement bundled interventions for Black women with HIV. Future research should examine the relationship between readiness and clinical outcomes.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-024-11568-x.

Contributions to the literature

• Adds to the limited empirical research focused on the utilization of bundled evidence-based and evidence-informed interventions to improve HIV care and treatment for priority populations.

• Illustrates that health service organizations consisting of health centers, AIDS service organizations, community-based organizations, and health departments are invested in providing comprehensive bundled interventions as depicted in high levels of organizational readiness scores.

• Organizational networks, specifically external partnerships, and collaboration through network weaving across silos; and leadership and organizational buy-in are central to facilitating implementation.

• Recommendations to advance HIV research and practice are provided.

Introduction

Despite scientific advances in HIV prevention and treatment, Black/African American women are overrepresented in the number of new HIV diagnoses and prevalence [1, 2], accounting for 54% of diagnoses of HIV infection among women compared to their White (24%) and Hispanic/Latino (17%) counterparts [3]. Black women with HIV have worse HIV-related health outcomes e.g., suboptimal pre-exposure prophylaxis (PrEP) uptake [4–6], and lower viral suppression [7] than women of other racial and ethnic subgroups. Social, community, and structural factors such as stigma, poverty, racism, discrimination, gender norms, mass incarceration, and violence against women contribute to and reinforce the disparate distribution of HIV among Black women and HIV care outcomes [8–12].

To date, a compendium of evidence-based and evidence-informed interventions have been developed to address gaps in the HIV care continuum [13]. Understanding how these evidence-based and -informed interventions are implemented including barriers and facilitators to intervention uptake, strategies that facilitate uptake, and adaptations and tailoring of interventions is essential for maximizing the impact interventions have for people with HIV [14]. Implementation science has emerged as an important field for researching and evaluating HIV/AIDS prevention and treatment interventions, policies, and practices to make substantial health impacts [15–19]. However, the evaluation of interventions in the HIV care continuum using implementation science is inconsistent and there is a need for optimizing implementation science in the use of evidence-based interventions [20, 21].

Given the multimodal barriers to accessing, engaging, and staying in care, particularly among Black women with HIV, implementing a set of evidence-based and or evidence-informed interventions [13] collectively could substantially improve linkages to care and supports, retention in care and health outcomes. Bundled interventions are a group of evidence-based practices that when implemented together [22], have the potential to address socio-cultural health needs, expand the delivery and utilization of comprehensive culturally responsive HIV care and treatment services, support ongoing engagement in care, and reduce HIV-related health disparities, particularly for Black women with HIV.

The goals of the 2022–2025 National HIV/AIDS Strategy aim to improve HIV-related outcomes of people with HIV and reduce HIV-related disparities and health inequities [23]. Aligned with these goals, the Black Women First (BWF) initiative (2020–2024) supports the implementation, and evaluation of bundled evidence-informed interventions for Black cisgender and transgender women with HIV to improve care and treatment and address these disparate outcomes [24]. The evaluation of the BWF initiative is guided by Greenhalgh’s Conceptual Model of Diffusion of Innovations in Health Service Organizations [25] and Proctor’s Implementation Outcomes Framework [26]. The initiative is implemented in accordance with the HRSA HAB Implementation Science [14] efforts to end the HIV epidemic. Details about the interventions, assessments, and study sites including supplementary online content are previously reported [24]. Within the conceptual and evaluation framework for the initiative’s multisite evaluation, the determinants that influence the implementation and client outcomes within and across organizations include innovation, assimilation, diffusion, system antecedents, readiness, outer context, and implementation process [24]. This study explores organizational readiness as a determinant in the implementation of the BWF initiative.

Despite the potential benefits of bundled interventions, understanding how best to prepare and support organizations to implement bundled and multicomponent interventions has not been explored. Readiness to implement such interventions including challenges for adoption and implementation at the provider and organizational levels are not fully understood. Understanding readiness to implement bundled interventions for Black women, especially at the height of the coronavirus disease 2019 (COVID-19) pandemic [27, 28], can inform individual and contextual factors necessary for implementing and how organizations and communities can pivot to meet the needs of this population. In addition, contexts informing readiness can be utilized to plan for scaling up and replicating evidence-informed bundled interventions to other contexts (e.g., populations, organizations, and clinical systems, etc.), benefit the quality of care for other priority populations, and broad-scale use beyond demonstration projects represented in the BWF initiative.

As part of the overall evaluation of the BWF initiative, it was important to first assess whether staff, providers, and organizational leaders were ready to implement bundled interventions focused on addressing the needs of Black women with HIV. Organizational readiness is a key determinant of the successful implementation of interventions [29–31]. A multifaceted construct, organizational readiness for change is defined as “the extent to which organizational members are psychologically and behaviorally prepared to implement organizational change” [32]. It hypothesizes that organizational members need change commitment (shared resolve to implement change) and change efficacy (shared belief in collective capacity to implement change) to successfully implement an intervention [31]. High organizational readiness for change should result in greater investment in the implementation effort including the ability to address obstacles and complexities that arise during the implementation process [31, 32]. Conversely, the organizational readiness for change dimensions may influence the programmatic change by defining prevailing conditions that facilitate change, such as having adequate resources, staff with attributes of adaptability and growth orientation, and an organizational climate that values innovation. As such, and in relation to an initiative primarily focused on Black women, it is essential that there be a collective effort within an organization to adequately prepare to implement new models of care (e.g., bundled interventions) designed to address disparate HIV care outcomes and advance health equity for this priority population. This paper is among the first to examine organizational readiness to implement bundled interventions designed to improve HIV care and treatment for Black women. We report on readiness for change longitudinally amongst organizational members (i.e., staff, providers, and leaders) in 12 demonstration sites participating in the BWF initiative.

Methods

Design

The mixed method study design utilized a quantitative longitudinal survey conducted at three time points (baseline, 6- and 12-month) and cross-sectional qualitative interviews conducted before the implementation of the bundled interventions at each organization. The data collection materials and evaluation plan were approved by the Institutional Review Boards for the Evaluation and Technical Assistance Provider at the University of Massachusetts, Lowell (UML), Boston University Charles River Campus (BU), and Boston University Medical Campus (BUMC) (H-41438) and the local Institutional Review Boards of the 12 participating study sites in accordance with their respective guidelines. Detailed information on study site protocol numbers is documented in the overarching study protocol [24].

Setting and intervention

This study was conducted in health service organizations participating in the Black Women First (BWF) initiative funded by the Health Resources & Services Administration (HRSA), HIV/AIDS Bureau (HAB), Ryan White HIV/AIDS Program Part F Special Projects of National Significance (SPNS) and Minority HIV/ AIDS Fund [24]. Twelve organizations (sites) comprising of health centers, AIDS service organizations, community-based organizations, and health departments participated in the demonstration project focused on improving care and treatment coordination for Black women with HIV. Participating sites implemented an intervention bundle consisting of 2 or more evidence-informed interventions selected from HRSA HAB prescribed six domains of evidence-based/evidence-informed interventions for improving health outcomes for people with HIV [33, 34]. These domains included: (1) Enhanced patient navigation, case management, or peer engagement; (2) Red Carpet Care experience to address barriers to HIV care; (3) Stigma reduction interventions; (4) Trauma informed care interventions; (5) Self-efficacy, health literacy, and resiliency interventions; and (6) Interventions to address intimate partner violence, sexual violence, or other behavioral needs. Detailed information on each intervention type is documented in the overarching study protocol [24].

Sample selection, recruitment, and data collection

The sample frame for quantitative and qualitative data was all staff (e.g., administrators, evaluators, quality improvement managers), providers (e.g., community health workers, peer navigators/advocates, patient navigators, case managers, social workers, psychiatrists, primary care providers) across 12 sites and relevant partners participating in the BWF initiative. Participants held various roles relevant to and/or had decision-making responsibilities around the implementation of the bundled intervention.

Quantitative: The Evaluation and Technical Assistance Provider (ETAP) administered a survey to staff and providers of the 12 sites using the Research Electronic Data Capture (REDCap) web application at baseline, 6- and 12-month intervals. The website link was sent to BWF initiative staff and provider participants via email which included information and documentation of consent for participation, confidentiality agreement, and the survey questions. The study was performed according to the Declaration of Helsinki’s ethical principles for research involving human subjects. Participants were provided with information about the study electronically and an opportunity to ask questions. Respondents completed an electronic consent statement indicating that they had fully read and understood the purpose of the study and procedures. Participants were informed that completion of the survey was voluntary and that they could withdraw from the study during survey completion without comment or penalty by closing the survey link. Participants were also informed that all data collected and processed would be kept anonymous, confidential, and stored on a password-protected database. Three email reminders were sent to encourage participation within the 45-day survey completion window. Participants were reminded of the survey using the BWF initiative newsletter, and during site-specific monitoring and all-site evaluation monthly calls. Survey data collection was completed from April 7, 2021 – June 14, 2021 (baseline), October 7, 2021 – November 22, 2021 (6-month) and April 7, 2022 – May 20, 2022 (12-month).

Qualitative: Pre-implementation interviews were conducted between January and April 2021 with the 12 organizations participating in the BWF initiative to assess how prepared organizations were to implement evidence-based/evidence-informed interventions for Black women with HIV. Two open-ended questions were included in the survey administered using the REDCap described above. These questions captured barriers (“What are the 2 main challenges that you have encountered when implementing [intervention] for Black women with HIV?”) and facilitators (“What has helped you with implementing [intervention] for Black women with HIV?”) to implementing bundled interventions for the BWF initiative and were asked at baseline, 6- and 12- month data collection intervals. Qualitative methods have been previously reported and described in detail [24, 27].

Measures

Organizational readiness

The extent to which site participants were “psychologically and behaviorally prepared to implement organizational change” (pg. 381) [32] i.e., bundled interventions was assessed using the validated 12-item Organizational Readiness for Implementing Change (ORIC) instrument [35]. Survey response items were directed to assessing change commitment (i.e., shared resolve to implement change; five items) and change efficacy (i.e., shared belief in their collective capability to make a change; seven items) [35] about implementing bundled evidence-informed and evidence-based for Black women with HIV in their organizations. The ORIC items are scored using a 5-point Likert scale with options ranging from “disagree” (1) to “agree” (5) with a total score of 12–60 and have demonstrated content consistency and adequacy [35]. A high score represents greater readiness to implement change. Quantitative data examined overall organizational readiness scores as well as change commitment and change efficacy subscales. The survey questionnaire is available as a supplemental document (Additional File 1).

Sociodemographic information

Data on personnel characteristics (i.e., respondent age, gender, race and ethnicity, education, job role, length of time in the role, and length of time in the organization) were collected at baseline, 6- and 12-month data collection periods.

Organizational characteristics

Organizational factors assessed included site/subcontractor role whether the organization had participated in a previous HRSA/SPNS project, organizational setting (e.g., health department, community-based organization, etc.), the annual number of patients/clients with HIV served in the organization, magnitude of organization’s service area, type(s) of other federal funding available to the organization, types of interventions in the bundled (2 or more), and types of priority population(s) served.

Analysis

Quantitative: Survey data collected using the REDCap web-based application were exported into Statistical Analysis Software (SAS, version 9.4) software for statistical analysis. Descriptive statistics were used to assess the demographics of staff and providers, organizational characteristics, and 12-item organizational readiness for implementing change (ORIC) scale. Specifically, descriptive individual ORIC items scored 1 to 5 were independently analyzed, with higher scores indicating greater change commitment and perception of efficacy to enact change. Overall organizational readiness was built by summarizing 12 individual ORIC items, while the change commitment and change efficacy subscales were assessed using the sum of five and seven individual ORIC items, respectively. Mean scores with standard deviations (SD) and median scores with interquartile range (IQR) measured levels of readiness. We tested three measures (overall organizational readiness, change efficacy, and change commitment subscales) at three time points (baseline, 6- and 12-month). The Kolmogorov–Smirnov test was used to assess the normality of the data. Associations between categories of participant and organizational level characteristics were assessed using linear models of ORIC data with clustering by site at baseline, 6- and 12-month. Cronbach’s alpha was used to measure the internal consistency of the ORIC domains.

Qualitative: Directed content analysis [36] was used to explore the context in which staff collectively resolved barriers and their capability to make changes to ensure the successful implementation of bundled interventions from the two open-ended questions in REDCap and pre-implementation baseline interviews conducted with site staff. Both open-ended survey responses and pre-implementation baseline interview Zoom-generated transcripts were managed using NVivo software version 12.0 [37]. Three researchers conducted thematic analysis and generated categorical codes for ORIC domains. Text segments from survey responses and interview transcripts were then coded in NVivo in accordance with the generated codes by ORIC domain. Coders met regularly to run coding comparison queries through NVivo to assess inter-rater reliability. Inter-rater reliability was determined by the percentage agreement statistical method. If the percentage agreement between two coders was below 80%, coding outputs were reviewed in detail and discussed to reconcile any discrepancies in agreement and validate the reliability of the themes identified.

Qualitative findings were used to further contextualize the quantitative findings. The research team met to review and discuss the quantitative results. Subsequently, they examined the qualitative findings to gain additional insights and add depth to their understanding of the quantitative data. A series of meetings were held in which members of both the qualitative and quantitative research teams discussed the findings together. As themes emerged from the quantitative data, the qualitative research team provided illustrative examples and quotes from the text that highlighted each point. The team synthesized these insights and wrote a comprehensive summary. They employed memos and short summaries, which were then transformed into the final findings.

Results

Recruitment and retention

Longitudinal data collection began in April 2021, and follow-up data collection ended in May 2022. Of the 136 staff and providers identified as participating in the BWF initiative across 12 organizations at the beginning of the study period, 97 completed the pre-implementation baseline survey, resulting in a 71.3% response rate. During follow-up periods, 109 out of 149 participants (73.2%) completed the survey at 6 months, and 59 out of 126 participants (46.2%) completed the survey at 12 months. The sample sizes and completion rates at baseline, 6- and 12-month periods are detailed in Table 1.

Table 1.

Sample and completion rates

	Baseline	6-month follow-up	12-month follow-up
Sampled (N)	136	149	128
Completed (N)	97	109	59
Response rate (%)	71.3%	73.2%	46.2%

Participant and organizational/agency characteristics

As shown in Tables 2 and 147 unique participants completed at least one survey during the study period. The mean age of participants was 44.3 (SD 12.0) and ranged from 22 to 74 years. The majority of respondents were women (83.7%). Over half of the participants identified as Black (57.8%), with the rest identifying as White (25.2%), Hispanic (8.2%), and multiracial or other (7.5%). Most participants had post-college or graduate education (69.4%). A third were administrators (34.7%), most had been working with the organization for five years or more (45.6%), and nearly half (49.0%) had been in their roles for over five years. The majority described themselves as staff (70.1%) rather than subcontractors (29.9%). Most participants (45.6%) had previous Special Projects of National Significance (SPNS) program/funding experience. A majority (61.2%) worked in community-based or health department organizations. Almost half (49.7%) belonged to organizations serving 0-500 clients with HIV annually, and the majority were from urban settings (85.0%). Additionally, a quarter (25.2%) of the organizations served 2–3 counties.

Table 2.

Participant and organization characteristics

Variables	Baseline (N = 97)		6-month (N = 109)		12-month (N = 59)		All unique participants (N = 147)
	Mean (SD) or n %		Mean (SD) or n %		Mean (SD) or n %		Mean (SD) or n %
Age at first data collection point (in years) (mean (SD))	44.7 (11.9)		43.8 (11.5)		46.8 (12.0)		44.3 (12.0)
18–34	22.0	22.7	27.0	24.8	10.0	17.0	33.0	22.5
35–44	29.0	29.9	36.0	33.0	18.0	30.5	51.0	34.7
45–54	25.0	25.8	24.0	22.0	12.0	20.3	30.0	20.4
55–64	15.0	15.5	17.0	15.6	15.0	25.4	23.0	15.7
65 and older	6.0	6.2	5.0	4.6	4.0	6.8	10.0	6.8
Gender
Man	9.0	9.3	8.0	7.3	6.0	10.2	14.0	9.5
Woman	84.0	86.6	91.0	83.5	52.0	88.1	123.0	83.7
Transgender and others	4.0	4.1	10.0	9.2	1.0	1.7	10.0	6.8
Race1
Black (non-Hispanic)	55.0	56.7	60.0	55.1	33.0	55.9	85.0	57.8
White (non-Hispanic)	25.0	25.8	30.0	27.5	21.0	35.6	37.0	25.2
Hispanic	9.0	9.3	11.0	10.1	2.0	3.4	12.0	8.2
Multiracial or other	7.0	7.2	7.0	6.4	3.0	5.1	11.0	7.5
Highest educational level
Post-college/Graduate	70.0	72.2	78.0	71.6	41.0	69.5	102.0	69.4
College graduate (4-year college or university)	10.0	10.3	13.0	11.9	8.0	13.6	18.0	12.2
Some college or under	17.0	17.5	18.0	16.5	10.0	17.0	27.0	18.4
Organizational role
Community health worker/Peer navigator/Advocate/Patient navigator	20.0	20.6	25.0	22.9	12.0	20.3	32.0	21.8
Case manager/ Social worker	10.0	10.3	12.0	11.0	4.0	6.8	17.0	11.6
Mental health/Primary care provider	11.0	11.3	10.0	9.2	5.0	8.5	16.0	10.9
Administrator	42.0	43.3	31.0	28.4	21.0	35.6	51.0	34.7
Evaluator/Quality improvement manager	14.0	14.4	19.0	17.4	12.0	20.3	21.0	14.3
Other	0	0.0	12.0	11.0	5.0	8.5	10.0	6.8
Length in this role2
< 1 year	17.0	17.5	16.0	14.7	10.0	17.0	31.0	21.1
1–5 years	29.0	29.9	41.0	37.6	17.0	28.8	43.0	29.3
> 5 years	50.0	51.6	52.0	47.7	32.0	54.2	72.0	49.0
Length in the organization
< 1 year	23.0	23.7	19.0	17.4	9.0	15.3	37.0	25.2
1–5 years	27.0	27.8	39.0	35.8	17.0	28.8	43.0	29.3
> 5 years	47.0	48.5	51.0	46.8	33.0	55.9	67.0	45.6
Affiliation
Staff	68.0	70.1	73.0	67.0	41.0	69.5	103.0	70.1
Subcontractor	29.0	29.9	36.0	33.0	18.0	30.5	44.0	29.9
Special Projects of National Significance (SPNS) experience
Yes	43.0	44.3	54.0	49.5	37.0	62.7	67.0	45.6
No	32.0	33.0	34.0	31.2	14.0	23.7	45.0	30.6
Don’t know	22.0	22.7	21.0	19.3	8.0	13.6	35.0	23.8
Organization type
Health center/Hospital	21.0	21.7	25.0	23.0	17.0	28.8	30.0	20.4
AIDS service organization	19.0	19.6	20.0	18.4	12.0	20.3	27.0	18.4
Community based organization/Health department/Other	57.0	58.8	64.0	58.7	30.0	50.9	90.0	61.2
Annual number of clients with HIV served
0–500	53.0	54.6	45.0	41.3	23.0	39.0	73.0	49.7
501–2000	26.0	26.8	38.0	34.9	22.0	37.3	42.0	28.6
2000+	18.0	18.6	26.0	23.9	14.0	23.7	32.0	21.8
Geographic setting
Urban	81.0	83.5	96.0	88.1	52.0	88.1	125.0	85.0
Suburban	10.0	10.3	9.0	8.3	4.0	6.8	14.0	9.5
Rural	6.0	6.2	4.0	3.7	3.0	5.1	8.0	5.4
Organization’s service area
1 county	23.0	23.7	25.0	22.9	18.0	30.5	33.0	22.5
2–3 counties	27.0	27.8	16.0	14.7	13.0	22.0	37.0	25.2
3–4 counties	12.0	12.4	19.0	17.4	8.0	13.6	21.0	14.3
5–6 counties	13.0	13.4	13.0	11.9	6.0	10.2	18.0	12.2
7–8 counties	6.0	6.2	9.0	8.3	3.0	5.1	7.0	4.8
9 + counties	16.0	16.5	27.0	24.8	11.0	18.6	31.0	21.1

This table represents unique participants completing a survey at any time point in the study period

¹Unknown/Missing for 2 participants

²Unknown/Missing for 1 participant

Organizational readiness for implementing change (ORIC) across the BWF initiative and by site

The result of individual organizational readiness assessment survey items (Likert scale from 1 = strongly disagree to 5 = strongly agree) indicates high levels of agreement regarding readiness to implement bundled interventions to improve care and treatment for Black women with HIV a median score of 5 across all items at baseline, 6-month and 12-month periods. As shown in Table 3, all individual ORIC items had a mean score of greater than 4.5 across each collection period. The ORIC item 12 “People who work here feel confident that they can manage the politics of implementing a bundled intervention to improving care and treatment coordination for Black women with HIV” had the lowest score in each period.

Table 3.

Statistics of individual Organizational readiness for implementing change (ORIC) items

		Baseline (N = 97)	6-month (N = 109)	12-month (N = 59)
ORIC Subscales	ORIC Question Item	Mean (SD)
Change Efficacy	People who work here feel confident that the organization can get people invested in implementing a bundled intervention to improving care and treatment coordination for Black women with HIV.	4.77 (0.53)	4.68 (0.59)	4.51 (0.80)
	People who work here are committed to implementing a bundled intervention to improving care and treatment coordination for Black women with HIV.	4.86 (0.43)	4.79 (0.45)	4.73 (0.58)
	People who work here feel confident that they can keep track of progress in implementing a bundled intervention to improving care and treatment coordination for Black women with HIV.	4.64 (0.60)	4.64 (0.66)	4.75 (0.51)
	People who work here will do whatever it takes to implement a bundled intervention to improving care and treatment coordination for Black women with HIV.	4.67 (0.59)	4.59 (0.75)	4.66 (0.63)
	People who work here feel confident that the organization can support people as they adjust to implementing a bundled intervention to improving care and treatment coordination for Black women with HIV.	4.70 (0.54)	4.58 (0.74)	4.61 (0.67)
	People who work here want to implement a bundled intervention to improving care and treatment coordination for Black women with HIV.	4.86 (0.43)	4.80 (0.57)	4.81 (0.47)
	People who work here feel confident that they can keep the momentum going in implementing a bundled intervention to improving care and treatment coordination for Black women with HIV.	4.62 (0.64)	4.55 (0.67)	4.54 (0.65)
Change Commitment	People who work here feel confident that they can handle the challenges that might arise in implementing a bundled intervention to improving care and treatment coordination for Black women with HIV.	4.58 (0.67)	4.55 (0.65)	4.59 (0.67)
	People who work here are determined to implement a bundled intervention to improving care and treatment coordination for Black women with HIV.	4.76 (0.52)	4.72 (0.59)	4.76 (0.57)
	People who work here feel confident that they can coordinate tasks so that implementing a bundled intervention to improving care and treatment coordination for Black women with HIV goes smoothly.	4.64 (0.63)	4.63 (0.69)	4.63 (0.67)
	People who work here are motivated to implement a bundled intervention to improving care and treatment coordination for Black women with HIV.	4.80 (0.47)	4.78 (0.57)	4.71 (0.64)
	People who work here feel confident that they can manage the politics of implementing a bundled intervention to improving care and treatment coordination for Black women with HIV.	4.48 (0.79)	4.47 (0.78)	4.56 (0.57)

Table 4 illustrates overall ORIC scores for sites participating in the BWF initiative. Across all participating sites, scores for overall organizational readiness for implementing change (ORIC) (mean 56.4, median 59, interquartile range [IQR] 5) and subscales of the ORIC change efficacy (mean 32.4, median 35, IQR 4), and change commitment (mean 24, Median 25, IQR 1), were high indicating greater overall readiness, commitment, and efficacy for change at baseline. The high scores remained consistent at 6-month [overall ORIC (mean 55.8, median 58, IQR 6)] and 12-month [overall ORIC (mean 55.9, median 58, IQR 5)] follow-up periods. All scores had high reliability (Cronbach’s alpha > 0.9) at each time point. Linear models of ORIC data with clustering by site at baseline, 6- and 12-month are shown in Additional Files 3, 4, and 5 respectively. Least squares means indicate that levels of overall organizational readiness, change efficacy subscale i.e., shared belief in capability to make change, and change commitment subscale i.e., shared resolve to implement change taking into account clustering by site. The high levels of readiness at baseline are consistent at 6- and 12-month follow-up periods. However, the data show no significant differences among all the calculated differences (age, role, duration in role, years worked in organization, and organization type) between the least squares means (LS means), after clustering by site, and adjusting for the multiple comparison tests using Tukey-Kramer adjustment at baseline (Additional File 2) and 12-months (Additional File 4). Results show significant difference in the association between staff role and overall organizational readiness (p = 0.007) and change efficacy (p = 0.006) and change commitment (p = 0.020) subscales at 6 month (Additional File 3).

Table 4.

Organizational Readiness for Implementing Change (ORIC) across the Black women First (BWF) initiative

Organizational Readiness for Implementing Change (ORIC) across all participating sites
	Overall ORIC a					Change Efficacy					Change Commitment
	Mean	SD	Median	IQR b	Cronbach’s alpha	Mean	SD	Median	IQR b	Cronbach’s alpha	Mean	SD	Median	IQR b	Cronbach’s alpha
All Sites
Baseline (N = 97)	56.4	5.3	59.0	5.0	0.9	32.4	3.5	34.0	4.0	0.9	24.0	2.1	25.0	1.0	0.9
6-month (N = 109)	55.8	6.3	58.0	6.0	0.9	32.1	4.0	34.0	5.0	0.9	23.7	2.6	25.0	2.0	0.9
12-month (N = 59)	55.9	6.2	58.0	5.0	0.9	32.2	3.9	34.0	4.0	0.9	23.7	2.6	25.0	1.0	0.9
Organizational Readiness for Implementing Change (ORIC) by site
	Overall ORIC a					Change Efficacy					Change Commitment
	Mean	SD		Median	IQR b	Mean	SD		Median	IQR b	Mean	SD		Median	IQR b
Site A
Baseline (n = 9)	53.2	8.1		54.0	10.0	30.4	4.9		29.0	7.0	22.8	3.4		25.0	3.0
6-month (n = 9)	53.6	6.2		55.0	4.0	30.6	3.7		31.0	3.0	23.0	2.6		24.0	2.0
12-month (n = 6)	58.8	1.6		59.5	2.0	34.2	1.2		34.5	1.0	24.7	0.5		25.0	1.0
Site B
Baseline (n = 6)	55.7	3.8		55.5	8.0	32.0	2.7		31.5	5.0	23.7	1.6		24.0	2.0
6-month (n = 15)	55.3	7.0		59.0	8.0	31.7	4.6		34.0	7.0	23.7	2.6		25.0	2.0
12-month (n = 6)	47.2	6.4		47.5	2.0	26.8	3.4		28.0	3.0	20.3	3.3		20.5	3.0
Site C
Baseline (n = 3)	60.0	0.0		60.0	0.0	35.0	0.0		35.0	0.0	25.0	0.0		25.0	0.0
6-month (n = 7)	59.3	1.5		60.0	1.0	34.3	1.5		35.0	1.0	25.0	0.0		25.0	0.0
12-month (n = 5)	60.0	0.0		60.0	0.0	35.0	0.0		35.0	0.0	25.0	0.0		25.0	0.0
Site D
Baseline (n = 13)	58.4	2.9		60.0	2.0	33.9	1.9		35.0	2.0	24.5	1.1		25.0	0.0
6-month (n = 15)	56.7	4.0		58.0	7.0	32.7	2.6		33.0	5.0	24.1	1.6		25.0	2.0
12-month (n = 14)	56.9	4.8		60.0	6.0	32.8	3.2		35.0	6.0	24.1	2.0		25.0	0.0
Site E
Baseline (n = 4)	55.8	1.0		55.5	1.5	31.5	0.6		31.5	1.0	24.3	0.5		24.0	0.5
6-month (n = 6)	54.7	4.0		55.0	5.0	32.3	2.4		31.5	4.0	22.5	2.1		22.0	4.0
12-month (n = 0)	--	--		--	--	--	--		--	--	--	--		--	--
Site F
Baseline (n = 6)	60.0	0.0		60.0	0.0	35.0	0.0		35.0	0.0	25.0	0.0		25.0	0.0
6-month (n = 6)	56.2	4.6		57.0	5.0	32.7	2.9		34.0	5.0	23.5	2.4		25.0	4.0
12-month (n = 5)	56.2	5.8		58.0	1.0	32.2	4.1		34.0	1.0	24.0	1.7		25.0	1.0
Site G
Baseline (n = 2)	53.0	9.9		53.0	14.0	29.0	8.5		29.0	12.0	24.0	1.4		24.0	2.0
6-month (n = 7)	57.4	6.8		60.0	0.0	33.6	3.8		35.0	0.0	23.9	3.0		25.0	0.0
12-month (n = 4)	59.5	1.0		60.0	1.0	34.5	1.0		35.0	1.0	25.0	0.0		25.0	0.0
Site H
Baseline (n = 22)	55.4	7.2		58.5	4.0	32.0	4.4		34.5	3.0	23.4	3.0		25.0	1.0
6-month (n = 12)	56.8	5.3		59.5	5.0	32.9	3.4		35.0	4.0	23.9	1.9		25.0	1.0
12-month (n = 5)	57.2	1.3		57.0	2.0	32.8	1.3		33.0	2.0	24.4	0.6		24.0	1.0
Site I
Baseline (n = 8)	55.9	6.4		59.0	6.5	31.9	4.7		34.0	5.0	24.0	1.8		25.0	1.5
6-month (n = 9)	51.0	14.0		57.0	11.0	29.1	8.2		32.0	7.0	21.9	5.9		25.0	4.0
12-month (n = 1)	60.0	--		60.0	0.0	35.0	--		35.0	0.0	25.0	--		25.0	0.0
Site J
Baseline (n = 8)	59.4	0.9		60.0	1.5	34.4	0.9		35.0	1.5	25.0	0.0		25.0	0.0
6-month (n = 8)	58.0	2.1		58.0	3.0	33.1	1.8		33.0	3.0	24.9	0.4		25.0	0.0
12-month (n = 2)	47.0	12.7		47.0	18.0	26.5	6.4		26.5	9.0	20.5	6.4		20.5	9.0
Site K
Baseline (n = 10)	54.3	2.6		54.0	3.0	30.8	1.5		30.5	2.0	23.5	1.9		24.0	1.0
6-month (n = 10)	53.6	4.8		54.0	8.0	30.3	3.1		30.0	4.0	23.3	2.0		24.0	4.0
12-month (n = 5)	58.2	2.2		59.0	3.0	33.4	2.3		35.0	3.0	24.8	0.5		25.0	0.0
Site L
Baseline (n = 6)	57.5	3.3		59.0	5.0	33.2	2.2		34.0	4.0	24.3	1.2		25.0	1.0
6-month (n = 5)	58.8	1.6		59.0	1.0	34.0	1.2		34.0	1.0	24.8	0.5		25.0	0.0
12-month (n = 6)	52.2	9.4		56.0	10.0	30.2	5.5		32.0	6.0	22.0	3.9		23.5	5.0

Footnotes:

^aORIC – Organizational Readiness for Implementing Change

^bIQR – Interquartile Range

Qualitative themes emerging from pre-implementation interviews highlight the context in which staff collectively resolved and the capability to make changes within the organization as a facilitator for successful implementation. The findings demonstrate organizational members’ motivation to change as reflected in planning and assessment efforts and organizational changes made to prepare for the implementation of bundled interventions.

Planning and assessment

During the planning and assessment phases teams described working collaboratively to both design programming to “uplift” Black women and to prepare for implementation. This included team coordination and the development of workflows to support implementation. During pre-implementation interviews, staff described the importance of their work and a commitment to supporting Black women and designing innovative programs to enhance their linkage to care as well as their social and economic well-being.

“The team is working and learning from each other. They are making sure to understand each other’s strengths and weakness and are feeling really confident about implementation. [In preparation they] have created workflows and flyers.”

“Our project proposes to use red carpet care services, health care community health workers and trauma informed care approach. … we saw combining all three will eliminate … the lag time … previously, we addressed barriers to care on a case-by-case basis, …bundling all three will decrease the length of time it takes from contact to actual linkage and overall retention in care for the span of the intervention.”

“… we started out as an organization that our focus was primarily on the Black gay men. We have developed our [women’s] program and what is innovative is being able to give women …[additional] programs that they can focus on in terms of Financial Literacy or focusing on themselves and entrepreneurship and things … to uplift themselves. Our program is going to be about building and strengthening the Black woman. … not just about HIV.”

Organizational change

Team members discussed organizational changes they had made to prepare for implementation. This included assessing internal policies related to outreach and engagement as well as identifying key community partners to support engagement efforts. In addition, teams described staff training initiatives designed to increase preparedness, comfort, and self-efficacy with respect to implementation and intervention delivery. The team also hired new staff in key roles to support implementation.

“One of the areas I believe we really wanted to focus on was building internal capacity within our own agency as well as partners around assuring that we’re delivering trans affirming care. From an organizational level where we’re looking at our policies and practices to the overall delivery of care and treatment, level of engagement, our outreach efforts through the various strategies and interventions.“

“We have had several opportunities to collaborate and be co-trained. The trainings have brought about a greater sense of self-efficacy and comfortability among staff working on this project to implement the interventions and engage clients in trauma-informed, gender-affirming ways.”

Barriers and facilitators to implementing bundled interventions

Upon completing the 12-ORIC items, participants identified barriers and facilitators to implementation through two open-ended questions (See Table 5). At the organizational level, barriers were associated with resource constraints and organizational structure characterized by silo-ing. Sites also described barriers with respect to workflows and activities with partner agencies, client engagement, and aligning with client priorities. In addition, the COVID-19 pandemic was described by sites as a pervasive challenge that impeded their ability to implement the bundle and reach clients. Facilitators included leadership and organizational buy-in, staff motivation, and partnerships. Staff capacity was seen as both a facilitator and barrier; a facilitator in that staff were motivated to support the intervention and barrier in cases where staff did not have the capacity to implement the intervention and when there was a high level of resource constraints related to staffing.

Table 5.

Barriers and facilitators to implementing bundled interventions

Barriers	Facilitators
“Lack of unity to work together to address HIV in the community and also having the ability to collaborate with other agencies that may provide similar services in the community.”	“Keeping the target population at the forefront of my mind. They need these services. That is my motivation to get this project off the ground and implemented.”
“Identifying areas to implement the interventions within the partner site that works seamlessly with current operations and developing a strong cross referral relationship that provides seamless transition to care.”	“Through my pain there has been a purpose revealed to have an impact on others who may experience similar life experiences. I also have been afforded with the leadership within to encourage, and support me in the development and implementation of bundled interventions.”
“I’ve encountered clients not really wanting to get involved in this program due to past experiences with other programs not actually delivering the ultimate goal on what the program was originally based on.”	“Establishing new relationships, trust, and strong bonds among leadership of ours and partner organizations has really helped us move the project along.”
“Coronavirus is currently our biggest challenge. Our Red-Carpet Care intervention had to be modified accordingly, and this makes it more challenging to schedule a date on which to hold our Trauma-Informed Care training.”	“We have had several opportunities for to collaborate and be co-trained. The trainings have brought about a greater sense self-efficacy and comfortability among staff working on this project to implement the interventions and engage clients in trauma-informed, gender-affirming ways.”
“Establishing a smooth flow within the agency to ensure the Black women being served have their specific needs met, and that the care teams are all involved in the intervention of the client together. We have a history of serving white gay men before merging into the wrap-around agency, so some stigma within the community has to be addressed in particular for marketing to black women.”	“Our amazing team members who are genuinely committed to improving the lives of clients, whether through bundled interventions or other means. We have gotten used to meeting regularly to work through issues together.”
“Accessing funding to sustain program services long-term.”	“Community partnerships, organizational capacity and expertise, leveraging existing and previous success to scaffold and strengthen programming.”

Discussion

Organizational readiness is a multidimensional and multilevel construct consisting of individual and organizational dimensions. To our knowledge, this is the first study to provide insight into organizational readiness to implement bundled interventions for Black women with HIV. Findings indicate that all 12 sites exhibited high levels of overall organizational readiness, change commitment, and change efficacy. There is a high level of agreement for both commitment and efficacy domains ranging from “Somewhat Agree” to “Agree” across each item in the ORIC subscales. The variation in overall organizational readiness, change efficacy, and change commitment by participating sites and staff roles at baseline and follow-up period respectively are of note. Considering clustering by site, mean scores of overall organizational, change efficacy, and commitment remain high at baseline, 6- and 12-month across individual and organizational characteristics. We found a significant association in the overall readiness, change efficacy, and change commitment by staff role only at 6 months. These results, however, should be interpreted with caution. We were unable to determine if the difference observed is due to changes in leadership buy-in and organizational capacity over time to hire and orient new staff in the bundled intervention or if existing staff improved capacity and comfort with the bundled intervention. Our results show no differences among all other individuals (age, years worked in the organization, duration in the role) and organizational (type of organization) characteristics. Future research should examine individual and organizational contextual factors related to staff roles and their influence on organizational readiness.

Qualitative data enabled us to have a more in-depth understanding of organizational readiness scores from the perspective of various providers across multiple agencies. Qualitative findings show that the outer context as well as the implementation process itself and system antecedents impact readiness. Organizational networks, specifically external partnerships, increased readiness for many sites. Implementation processes characterized by collaboration through network weaving across silos within the organization and communications systems that engage external partners, as well as resources available for hiring and training, supported readiness. Our study also highlights the role of system antecedents, such as staff perception of whether an intervention aligns with their desired needs and lack of unity with partner agencies around defined community goals, can also be barriers to intervention uptake. Employing implementation strategies that cultivate staff and partner agency buy-in, in addition to training staff, is key for the uptake of evidence-based interventions. Future research could examine multilevel systems factors influencing readiness for implementing bundled evidence-based interventions.

While organizational readiness is often assessed during pre-implementation, our study is unique in that it evaluates readiness during the first 12 months of the project. While unconventional, the longitudinal data allowed the study team to assess readiness particularly given that all 12 sites embarked on the initiative at the height of the COVID-19 pandemic, a time when individuals and organizations were required to make tremendous adjustments to organizational standard operating procedures and deal with staffing shortages and resource constraints. Our prior work has examined the implementation of the Black Women First (BWF) initiative during the COVID-19 pandemic [27, 28]. Measuring readiness longitudinally allows for the assessment of readiness for established and new organizational members. It is important to understand people’s readiness to implement a particular type of intervention upon hire so that appropriate adjustments can be made to increase their readiness to level up with established staff and prevent disconnects in the implementation team and unanticipated implementation challenges and barriers. Repeated assessments are an opportunity to identify challenges and opportunities that may have been missed in prior assessments, changes in organizational climate about the intervention, or other contextual factors in the intervention/program life cycle (e.g., staffing changes, funding changes, competing interventions that have a relative advantage, etc.). Assessing readiness throughout the entire implementation allows stakeholders to see progress towards being ready over time and may potentially be more reflective about sustained organizational readiness and anchor points for long-term success. Our results demonstrate that despite the challenges experienced in ramping up interventions at the height of the COVID-19 pandemic, overall readiness and readiness subscales (change efficacy and change commitment) were consistent throughout the BWF initiative. Future research should explore the relationship between readiness, sustainability, and the sustainment of interventions and programs, as well as how the readiness of organizational members, considering their various characteristics, impacts health and social outcomes for priority populations.

Key strategies for sharing and utilizing organizational readiness findings during the initiative

The results of the readiness assessment were disseminated throughout the initiative and utilized by the ETAP and demonstration sites to guide implementation and evaluation processes. The ETAP adopted a comprehensive approach to share the findings with site staff, organizational leadership, evaluation partners, and funding partners to bolster the implementation processes across the initiative.

• (A)Monthly monitoring meetings with sites: Readiness findings were used to identify opportunities for technical assistance and to tailor support to meet the specific needs of each site.
• (B)Bi-monthly intervention cohort sessions: ETAP coaches and intervention site staff met in bi-monthly meetings throughout the initiative to identify challenges and generate solutions to intervention implementation and to strategize and develop technical assistance tools and supports for specific sites and the broader BWF initiative. The bi-monthly cohort meetings provided opportunities for shared learning and support beyond the organization which may contribute to readiness to implement the intervention.
• (C)Bi-annual multi-site convenings: The ETAP presented readiness findings to facilitate a comprehensive discussion on the readiness findings, including the quantitative domains and constructs (change efficacy and change commitment), and to identify barriers and facilitators to implementation. Multisite convenings were an important avenue for these discussions because of the wide and diverse range of participants including site staff, organizational leadership, partner organizations, advisory board members, HRSA/HAB, and ETAP representatives. The diverse group of stakeholders discussed (1) site-specific and initiative-wide readiness, (2) strategies to overcome barriers to implementation and challenges with evaluation, (3) how to leverage opportunities to enhance the recruitment, engagement, and retention of Black women in the initiative, (4) strategize on meeting the needs to Black women during and beyond the initiative, and (5) ensure that organizational leaders who were not involved in monthly and bi-monthly meetings could understand and support the implementation efforts. These convenings played a crucial role due to the inclusion of a wide range of participants, fostering a comprehensive understanding of the initiative’s progress and challenges. They provided a platform for in-depth discussions on how to address obstacles and capitalize on opportunities, ensuring that all stakeholders could contribute to and support the implementation efforts effectively.

The ETAP’s multi-pronged approach in sharing readiness findings through monthly monitoring meetings, bi-monthly intervention cohort calls, and bi-annual multi-site convenings ensured that the implementation processes were well-supported and informed. This strategy facilitated continuous improvement and alignment across the initiative, ultimately enhancing the efforts to provide care and treatment to Black women.

Limitations

This is based on a large convenience sample and therefore study findings may not be generalizable and causal associations between variables should be interpreted with caution. Participating sites applied for and received funds to provide comprehensive services for Black women with HIV, and as such, may have been prepared and had the resources and infrastructure to implement culturally relevant and person-centered bundled interventions for Black women with HIV. While organizational readiness was assessed during pre-implementation (baseline), this study also measures readiness after implementation of the BWF initiative at three time points. Staff responses post-implementation were likely affected by experience, built infrastructure, and invested efforts to implement bundled interventions. Low response rates during the follow-up period increased the risk of non-response bias. Additionally, relatively small sample sizes across sites, variability in staffing roles, and variability of bundled interventions selected limits comparisons and findings should be interpreted with caution. Nonetheless, we included those variables that we considered most likely to explain organizational readiness to implement change. Despite these limitations, this study contributes to the limited knowledge of how prepared organizations were to implement bundled interventions to improve care and treatment for Black women across various organizational types, especially during the height of the COVID-19 pandemic.

Conclusions

In summary, this study provides valuable information about levels of organizational readiness to provide bundled interventions to improve care and treatment for Black women, and facilitators and barriers to implementing bundled interventions in health service organizations. The identification of these factors and the assessment of organizational readiness for change can inform the design and implementation of subsequent intervention efforts including avenues for implementation facilitation, opportunities for training and technical assistance, and shared problem-solving and decision-making. Future research should examine the relationship between organizational readiness and HIV care and treatment outcomes.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Abbreviations

BWF

Black Women First

ETAP

Evaluation and Technical Assistance Provider

HAB

HIV/AIDS Bureau

HRSA

Health Resources & Services Administration

ORIC

Organizational Readiness for Implementing Change

REDCap

Research Electronic Data Capture

SPNS

Special Projects of National Significance

Author contributions

AWW & LSM contributed to the conceptualization, funding acquisition, investigation, methodology, formal analysis, visualization, supervision, and validation of the manuscript. MPM, XZ, MR, CAC, & JND contributed to the formal analysis, methodology, and visualization. SR contributed to the funding acquisition, investigation, methodology, and supervision. HJC contributed to the investigation, methodology, and supervision. EJ contributed to the methodology, TD contributed to the visualization, JCS contributed to the investigation and methodology, and AD contributed to the funding acquisition of the manuscript. AWW, LSM, and MPM wrote the original draft of the manuscript. All authors contributed to the review and editing of the manuscript and approved the final version.

Funding

This study is funded by the Health Resources and Services Administration (HRSA) of the U.S. Department of Health and Human Services (HHS) grant #U90HA39727. The contents are those of the author(s) and do not necessarily represent the official views of nor an endorsement, by HRSA/HHS, or the U.S. Government.

Data availability

Data are not publicly available. Data associated with this article can be requested by contacting the corresponding author.

Declarations

Ethics approval and consent to participate

This study protocol was approved by the Institutional Review Boards for the Evaluation and Technical Assistance Provider at the University of Massachusetts, Lowell (UML), Boston University Charles River Campus (BU), and Boston University Medical Campus (BUMC) (H-41438), and the local Institutional Review Boards of the 12 participating study sites in accordance with their respective guidelines. The study was performed according to the Declaration of Helsinki’s ethical principles for research involving human subjects. Before completing the electronic survey, the Evaluation and Technical Assistance Provider (ETAP) obtained informed consent to participate from all of the participants.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.