Increasing acceptability and uptake of voluntary male medical circumcision in Zambia: implementing and disseminating an evidence-based intervention

Abstract

Voluntary medical male circumcision (VMMC) uptake in Africa could prevent 3.4 million HIV infections across a 10 year span. In Zambia, however, ~80 per cent of uncircumcised men report no interest in undergoing VMMC. The Spear & Shield (S&S) intervention has been shown to be more effective than control or observation of only conditions at increasing the number of VMMCs. This study identified predictors of S&S implementation success or failure to create an “early warning” system to enable remedial action during implementation. Participants were n = 48 staff members from 12 community health facilities conducting the S&S program in Lusaka Province, Zambia. Quantitative assessments included demographics, provider attitudes, barriers to research uptake, staff burnout, and organizational readiness. Qualitative interviews were also conducted and quantified for analysis using the Consolidated Framework for Implementation Research (CFIR). Two-thirds (66%) of staff were women with a mean age of 37.67 years (SD = 7.51). Quantitatively, staff performance (p = .033) and decreased levels of staff burnout (p = .025) were associated with S&S implementation success. Qualitatively, constructs such as improved planning, executing, and self-reflection and evaluation were associated with S&S implementation success (p = .005). Identifying these factors facilitated remedial action across health facilities. This study illustrates the utility of the CFIR to guide program decision making in VMMC implementation in the Zambian context. Early identification of challenges to implementation may enable remedial action to enhance the likelihood of program sustainability. Effective monitoring strategies for HIV prevention interventions may thus enhance dissemination, implementation, and sustainability goals to bridge research and practice.

Clinics in Zambia offering voluntary male medical circumcision to reduce HIV risk can be effectively monitored to achieve their goals to maintain the male circumcision program.

Implications

Researchers: Effective interventions for HIV prevention require effective monitoring to achieve dissemination, implementation, and sustainability goals and bridge the gap between research and practice, and this study demonstrates how to effectively use the Consolidated Framework for Implementation Research to facilitate these factors.

Practitioners: Interim analyses highlighted staff performance and attitudes as key components of implementation, and suggest that performance evaluation with remedial feedback was especially important when more frequent coaching is not feasible.

Policymakers: In an era of diminishing financial support, an “early warning” system based on the Consolidated Framework for Implementation Research, such as the one developed in this study, could more effectively implement interventions and achieve sustainable clinical services.

INTRODUCTION

The uptake of voluntary medical male circumcision (VMMC) in Africa could prevent 3.4 million new HIV infections and avert up to 300,000 HIV/AIDS-related deaths across a 10 year span [1]. In Zambia, HIV prevalence is ~19 per cent [2], whereas VMMC acceptability is low; ~80 per cent of uncircumcised men report no interest in undergoing VMMC (Zambia Sexual Behavior Survey, 2013). As of 2015, Zambia had achieved 56 per cent of the Zambian Ministry of Health’s goal of circumcising 80 per cent of eligible Zambian men by 2015 (National AIDS Council, http://www.nac.org.zm/article/zambia-scoops-award-vmmc-crusade), and the goal was recalibrated for achievement by 2020 [3]. This delayed attainment of VMMC program goals, despite subsidized program support, highlights the need to examine individual, structural, and operational hurdles associated with VMMC provision and uptake to optimize program delivery.

The Spear & Shield (S&S) project, a cluster randomized controlled clinical trial aimed at promoting acceptability and uptake of VMMC in Zambia, more than doubled the number of VMMCs performed in comparison with a control condition (adjusted odds ratio 2.45, 95% CI 1.24–4.90; p = .02) and provided more than eight times the number of VMMCs in comparison with an observation only condition [4]. Currently, the S&S program is being disseminated and implemented in the four Zambian Provinces with the highest rates of HIV. The Consolidated Framework for Implementation Research (CFIR) [5, 6] is being used to evaluate implementation effectiveness [5]. The CFIR assesses a variety of implementation domains, including intervention characteristics, outer setting, inner setting, characteristics of individual, and process, and can be utilized to identify factors that may influence or predict implementation success or failure as a self-correcting strategy to enhance program sustainability. In addition, staff and provider attitudes toward evidence-based practices (EBPs) [7, 8]; organizational characteristics [9, 10]; staff burnout [11, 12]; and readiness for or resistance to change, perceived capability, intervention fit, and leaders’ commitment to the intervention [13] are also utilized as predictors of implementation success [14].

Although the CFIR is a well-established framework for implementation science methodologies due to its ability to address the multilevel complexities, barriers, and facilitators of implementation, it has not been used in the context of HIV prevention, despite recommendations to use more novel methods to improve the adoption of evidence-based interventions in this context [15]. Given the importance of rapid uptake of VMMC in Zambia and S&S’s success in increasing the number of VMMCs performed, this study is proposed to identify predictors of program implementation success or failure, guided by the CFIR framework, in order to create an “early warning” system enabling remedial action during all stages of implementation, particularly early in implementation. It was theorized that creation of a system identifying potential weaknesses in the implementation process could be used as a model for dissemination and implementation programs and to enhance program sustainability.

METHODS

Prior to study onset, approval was obtained from the Research Ethics Committee of the University of Zambia School of Medicine and the University of Miami Institutional Review Board. Provincial and district level approvals were also obtained.

Study design, participants, and procedures

This study used a mixed methods design to overcome the methodological challenges of single-method studies in implementation science research [16]. Participants were 48 staff members [clinic officers, sisters in charge, professional nurses, counselors, and lay health workers, which included VMMC providers and voluntary counseling and testing (VCT) staff] drawn from 12 community health centers and hospitals conducting the S&S program in Lusaka Province, Zambia. Candidates for study participation at each of the 12 participating health facilities were identified by study staff in collaboration with clinic leadership. Candidates were employed at the clinic and knowledgeable regarding the S&S program and were provided with information about the study in detail in a private office and invited to participate. All candidates agreed to participate and provided informed consent.

Information was collected in both quantitative and qualitative formats; assessments were completed in two sessions to minimize participant burden. The qualitative component was comprised of an in-depth semistructured interview; the quantitative component consisted of implementation-related measures presented using Audio Computer-Assisted Self-Interview (ACASI) software. Participants were compensated 100 Kwacha ($10 USD) for their time and participation for each of the two study components. Qualitative and quantitative data from all health facilities were collected from June 16, 2016 to July 27, 2016.

S&S program

The current study is scaling up the evidence-based intervention, the S&S program, across 96 health facilities in Zambia. The S&S program consists of two components: (a) training and provision of the S&S intervention for HIV lay counselors and nursing staff targeting “hard to reach” Zambian men and their partners and (b) training for healthcare providers in the provision of VMMC [4]. Candidates for S&S intervention training or VMMC training at each of the 12 participating health facilities were identified by district health officers and clinic and hospital officers. The two components of the S&S program have been described in detail in previous research [4]. The aim of the current study, to describe the preliminary implementation and dissemination of the S&S program to clinics in Zambia, is summarized in Fig. 1, which illustrates the S&S implementation and dissemination.

Fig 1.

| Implementation and dissemination of spear & shield.

S&S program training

S&S intervention training

The S&S intervention component is a behavioral intervention that consists of four 90 min-manualized sessions delivered to men and women in gender concordant groups (~8–10 participants per group). The S&S training begins with a 2 day intensive training program on the manualized four-session intervention, followed by supervised practice of three “on-the-job” four-session groups with senior S&S trainers. Emphasis is placed equally on training techniques and intervention content. The first “on-the-job” (OJT) four-session group is led by the senior S&S trainer with the health facility staff person serving as an observer. The second OJT four-session group is co-led by both S&S and health facility staff person. The third OJT group is led by the health facility staff person with the S&S senior trainer as an observer. Thereafter, to ensure fidelity of the intervention, 10 per cent of subsequent sessions at each health facility are randomly selected for review by the Zambia coordinator and addressed with the U.S. investigators. As such, all S&S program sessions at each of the health facilities were audio recorded. The S&S intervention training also utilizes a training of trainers’ strategy such that trained health facility staff (a) recruit men and women into the S&S intervention, (b) lead S&S intervention sessions, and also (c) provide S&S intervention training to other healthcare staff.

VMMC training

At each health facility, three healthcare providers (physicians, nurses, and clinic officers) are trained to perform VMMC. All sites are provided with reusable surgical kits and a 3 month supply of starter kits of medical consumables, enabling all study sites to offer VMMC [4]. VMMC training is provided over a 2 week period at the University of Zambia School of Medicine; on-site supervision is provided in accordance with VMMC training guidelines [17].

Quantitative assessments

The interviews and ACASI assessments collected at each health facility were conducted with four health facility staff volunteers per site.

Demographics

Demographic information consisted of occupational titles, time in position, gender, age, education level, and monthly income.

Provider attitudes

Provider attitudes towards adoption of EBPs were assessed using an adapted version of the 15-item Evidence-Based Practice Attitude Scale (α = 0.73) [7]. Greater scores on this scale indicate greater intuitive appeal of adoption of EBPs, the likelihood of adopting EBPs, openness to new EBPs, and perceived divergence of EBPs.

Barriers to research uptake

Barriers to intervention uptake were measured using an adaptation by Funk, Champagne [9] of the 28-item Barriers to Research Practice Scale (α = 0.91). Greater scores on this scale indicate more barriers for uptake by the adopter, the organization, the innovation, and internal communication.

Staff burnout

Staff burnout was assessed using the Copenhagen Burnout Inventory [12]. Greater values on the scale of 18 items indicate greater levels of personal, work-related, and client-related burnout (α = 0.87).

Organizational readiness

Readiness for organizational change was measured using the 44-item Readiness for Organizational Change scale [13], consisting of 44 items, which evaluates intervention appropriateness, management support, change efficacy, and personal benefit of the intervention (α = 0.94). Higher scores indicate greater acceptance of practices that may change the organizational structure.

Qualitative assessments

S&S staff performance

Staff performance was qualitatively evaluated by one S&S project coordinator based on quality assurance (QA) checks of S&S sessions (job knowledge), work product (number of sessions completed), employee dependability (absenteeism), adaptability and team work, and level of supervision needed to complete tasks (QA) using biweekly progress reports. Staff performance evaluations were conducted to maintain frequent feedback loops between the project manager and staff to enhance implementation [10]. In the current study, staff were categorized as low-performing or high-performing based on the overall assessment of the employee by the project coordinator. For purposes of comparison, low-performing employees were given a value of 1 and high-performing employees were given a value of 0.

Interviews

Study personnel conducted qualitative, individual, in-depth, semistructured interviews in English. Rapport was developed during interviews with health care staff participants using casual introductions and conversation regarding general activities. The interview guide questions were developed collaboratively by USA and Zambian researchers based on previous work [4], and designed to parallel the CFIR model [5, 18]. Questions and stems are presented in Table 1. Interviews were an average of 17.5 min (range = 10.5 to 34.2 min). Interviews were audio-recorded by the Zambian team and transcribed verbatim by the U.S. team.

Table 1.

Interview questions and stems for qualitative interviews

1. Describe the training you received to provide VMMC or S&S services to clients. [Inner Setting]    Prompts: How was it provided to you? What were the most useful aspects of your training? When was the last training? Do you wish you had received more training? Why? How do you feel about the training you have received? What kind of ongoing training have you received? Have you received any additional training? Who provided the training? 2. Describe the VMMC/ S&S services at your clinic. [Characteristics of Individuals]    Briefly describe the VMMC/S&S services and your role, if any, in providing these services. How closely are the VMMC/S&S services related to the HCT/VCT services? What do you believe to be the goals of the VMMC/S&S programs? What are staff attitudes about providing VMMC/S&S services? What challenges are there in providing VMMC/S&S services? 3. Describe the environment at your clinic. [Inner Setting]    Prompts: How well do staff work together? Describe the relationship that exists between staff members, and between staff and supervisors. Describe the space available for VMMC/S&S services. Is the space sufficient to meet the needs of these programs? Are the necessary personnel or equipment (VMMC surgical kits) usually available when patients request such services? 4. For men considering VMMC, what efforts are made to get their partners involved? [Outer Setting] 5. Describe some of the challenges experienced by staff in implementing the VMMC services? What do you think is the most challenging aspect of the VMMC program? [Process] 6. What changes (if any) would you recommend to ensure that the guidelines for VMMC programs are followed? [Interviewer: Now please ask about S&S, replacing VMMC with S&S]. [Characteristics of Individuals]    Prompts: Describe what you think health care providers could do to improve implementation of VMMC services and achieve the best outcomes. Describe what you think the clinic leadership could do differently to enhance implementation of the VMMC programs and achieve the best outcomes. Describe what you think the community leadership could do to enhance implementation of VMMC services to achieve the best outcomes. Who are other partners, groups or agencies that should be involved in helping to improve VMMC services? What can they do to help make the program better? 7. What are some of the barriers (if any) that you feel prevent making changes to improve these programs? [Characteristics of Individuals]    Questions: What would need to be in place for these changes to happen and work well? How ready are staff to change if it would improve VMMC/S&S services? What would need to happen to help staff get ready for change? 8. What role (if any) did you play in initiating VMMC/S&S services at your clinic? [Characteristics of Individuals] 9. What are influential individuals (e.g., community leaders) saying about VMMC/S&S services? Who are these influential individuals? [Innovation Characteristics] 10. What steps have been taken, if any, to encourage staff members to commit to supporting VMMC/S&S services? [Process] 11. What does your clinic do to advise the community that you have VMMC/S&S services? [Outer Setting] 12. Do you receive any information about how well the VMMC/S&S services at your clinic are being accepted by the clinic patients? Or by the other clinic staff? [Outer Setting] 13. Is there anything you would like to add or would be useful to know to improve implementation of VMMC/S&S services?

VMMC Voluntary Medical Male Circumcision; S&S Spear & Shield; HCT HIV Counseling and Testing; VCT Voluntary Counseling and Testing.

S&S intervention uptake at health facilities

Uptake of the S&S behavioral intervention was defined as the number of session contacts for each of the clinics, as summarized in Table 2. Each health facility provider was given a goal of offering 24 sessions—three separate groups of men attending four S&S sessions, and three separate groups of women attending four S&S sessions. As illustrated in Table 2, some health facilities achieved initial sustainability and completed 40 sessions (two additional groups of men, two additional groups of women, Groups 4 and 5). Sites were ranked by the number of session contacts.

Table 2.

Spear & Shield intervention uptake: male and female session contacts by group and health facility

Health Facility	Group 1a		Group 2a		Group 3a		Group 4a		Group 5a		Total M/F					Rankb
	M	F	M	F	M	F	M	F	M	F	M	F	Total	% M	% F
Facility 1	38	27	34	44	29	29	25	29	30	42	156	171	327	48%	52%	1
Facility 2	21	42	29	32	28	34	36	32	26	23	140	163	303	46%	54%	2
Facility 3	33	30	26	34	27	37	48	38	0	0	134	139	273	49%	51%	3
Facility 4	35	29	30	25	33	33	26	28	0	0	124	115	239	52%	48%	4
Facility 5	29	34	23	35	27	37	0	41	0	0	79	147	226	35%	65%	5
Facility 6	20	26	29	32	29	44	0	31	0	0	78	133	211	37%	63%	6
Facility 7	19	20	23	41	24	26	23	28	0	0	89	115	204	44%	56%	7
Facility 8	25	19	26	32	28	26	0	24	0	13	79	114	193	41%	59%	8
Facility 9	28	28	25	34	20	26	0	0	0	0	73	88	161	45%	55%	9
Facility 10	27	33	13	40	0	32	0	0	0	0	40	105	145	28%	72%	10
Facility 11	31	35	29	27	0	9	0	0	0	0	60	71	131	46%	54%	11
Facility 12	32	17	37	27	0	0	0	0	0	0	69	44	113	61%	39%	12
Total	338	340	324	403	245	333	158	251	56	78	1121	1405	338
Mean	28.2	28.3	27.0	33.6	20.4	27.8	13.2	20.9	4.7	6.5	93.4	117.1	210.5
SD	6.1	7.3	6.1	5.8	12.7	12.3	17.5	16.1	10.9	13.3	36.1	37.1	67.4

^aValues represent the total number of session contacts across four Spear & Shield intervention sessions.

^bFacilities were ranked by the total number of sessions contacts.

Coding and quantification of qualitative interview transcripts

Following transcription of qualitative interviews, interview transcripts were coded and content-analyzed line-by-line using the CFIR matrix [5, 18], which consists of 41 different constructs across five broad domains associated with effective implementation of innovations. The five domains are intervention characteristics, outer setting, inner setting, characteristics of individual, and process. The intervention characteristics domain includes constructs such as evidence strength and quality, and complexity, which evaluate stakeholders’ perceptions of the intervention, including the quality and validity of the intervention, and the perceived difficulty of implementing the intervention, including its duration and disruptiveness in a specific setting. The outer setting domain includes constructs such as the needs and resources of those served by the health facilities, as well as external policy and incentives, which broadly measure the degree to which the needs of patients are known by the health facilities, including barriers and facilitators, are known and prioritized by the health facilities. The inner setting domains include constructs such as structural characteristics, namely the architecture, age, and size of the health facilities, and culture, including the norms and values of a particular health facility. The fourth domain of the CFIR framework is characteristics of individual, which include knowledge and beliefs about the innovation, including individuals’ knowledge and familiarity with the facts and principles of the intervention, and self-efficacy, or staff’s beliefs in their own capabilities to implement the intervention. The process domain includes constructs such as planning—diagnostic assessments to implement the intervention—and executing—actually implementing the intervention according to plan.

The CFIR matrix is specifically designed to code qualitative data and coding tools are provided by the authors to facilitate a consistent method of coding across studies [5, 18]. The coding matrix provided by the authors was used in its original form in the present study to maximize generalizability. Initially, two internal coders coded five interview transcripts to establish a coding strategy and develop a coding protocol specific to the material and data and collected by the study. Then, five external coders were trained on the coding strategies and were asked to code the five previously coded transcripts to assess the level of agreement and reliability of identified themes and interpretations. Ultimately, all statements made by each participant were coded into each of the 41 constructs of the CFIR matrix, resulting in 48 completed CFIR matrices, 1 matrix of 41 constructs for each participant. Coding disagreements between pairs of coders were on average 14.7 per cent (SD = 2.08). Disagreements were discussed until consensus was reached, and a final coded version of the CFIR matrix was generated for each participant. Following the training period, agreement between raters was excellent (κ = 0.85). In addition, frequent meetings were conducted with the team to discuss coded content and reflect how personal beliefs, values, perceptions, and assumptions may have influenced coding. Once all transcripts were coded, valence codes on a scale of −2 to 2 were assigned to each of the statements such that negative scores indicated a barrier to the construct and positive scores indicated a facilitator. Neutral statements were assigned a score of 0, and mixed statements were coded as −1 or 1 if statements were mostly negative or mostly positive, respectively. Valence codes were then totaled, which resulted in a score for each participant for each of the 41 constructs; CFIR construct scores within each CFIR domain were summed, resulting in 5 scores for each participant at each of the health facilities: Intervention characteristics, outer setting, inner setting, characteristics of individuals, and process, which were used as measures of implementation effectiveness. Given that the objective of the study was assessing and addressing challenges arising early in the implementation process, evidence of data saturation was not an aim of this exercise. Nevertheless, previous studies have concluded that data saturation is reached within 6 to 17 interviews, indicating that the sample size for this study is sufficient for data saturation for the first stage of implementation, despite ongoing dissemination efforts [19, 20].

Statistical analyses

Univariate statistics were used to describe demographic and organizational characteristics of staff. Using the number of session contacts, health facilities were categorized as “Bottom 4” and “Top 4” health facilities for comparison of sociodemographic characteristics, CFIR domains, and organizational functioning (Table 2).

Bivariate analyses, chi-squares and t-tests, were used to compare sociodemographic characteristics by clinic groups. Then, a series of generalized linear mixed models (GLMMS) were used to compare outcomes of interest (staff performance, CFIR constructs, and organizational functioning) between the “Bottom 4” and “Top 4” health facilities. To model the dependency of staff within health facilities, a heterogeneous compound symmetry covariance matrix structure was used. For proportions, a binomial distribution and a logit link were specified. Results from GLMMS were followed by least square mean comparisons to test for differences between “Bottom 4” and “Top 4” health facilities.

SAS 9.4 was used for all analyses. A threshold of p < .05 was used to determine statistical significance.

RESULTS

S&S intervention uptake, session contacts by health facility

On average, there were M = 93.4 (SD = 36.1) session contacts with men, and M = 117.1 (SD = 37.1) session contacts with women (total M = 210.5, SD = 67.4), a session contact here defined as a person attending a session, such that one person could attend four groups in one session, resulting in four session contacts. The majority of facilities met the goal of offering S&S sessions to three groups of men and women each, though most of the bottom 4 clinics did not. Overall, there were more session contacts by women than men (M = 117.08 [SD = 37.07] vs. M = 210.50 [SD = 67.42]). See Table 2 for a summary of session contacts by gender, group, and health facility.

Sociodemographic characteristics of participants

The subsample of participants for the current study was n = 32 men and women working at the bottom 4 and top 4 health facilities, as ranked by the number of session contacts. The “middle” performing health facilities were excluded to contrast between the bottom 4 and top 4 health facilities [6]; only the characteristics of the “top” and “bottom” performing clinics were relevant in the current study. Slightly more than half of the subsample staff had upper-level positions (clinic officers, sister in charge, or professional nurse; 56%). Approximately two-thirds (69%) had been in their position for more than 5 years, were female (66%), and had a mean age of 37.67 years (SD = 7.51). Nearly all (94%) had a diploma, certificate, masters, or doctorate degree, and two-thirds had a monthly income of more than 3400 Kwacha (USD ~$350). See Table 3 for details.

Table 3.

Health facility staff characteristics by bottom 4 versus top 4 health facilities

	All Mean (SD) (n = 32)	Bottom 4 Mean (SD) n (%) (n = 16)	Top 4 Mean (SD) n (%) (n = 16)	χ 2/t, p
Job title
Sister in charge or professional nurse	18 (56.3%)	7 (43.8%) 9	11 (68.8%)
Assistant nurse, lay health worker, counselor	14 (43.8%)	(56.3%)	5 (31.3%)	2, 03, 0.154
Time in position
Less than 5 years	10 (31.3%)	6 (37.5%) 10	4 (25.0%)
More than 5 years	22 (68.8%)	(62.5%)	12 (75.0%)	1.39, 0.238
Gender
Male	11 (34.4%)	6 (37.5%) 10	5 (31.3%)
Female	21 (65.6%)	(62.5%)	11 (68.8%)	0.14, 0.710
Age	37.66 (7.51)	37.38 (8.41)	37.38 (6.75)	0.28, 0.836
Education
Up to grade 12	2 (6.3%) 30	2 (12.5%) 14	0 (0.0%) 16
Diploma, certificate, masters, doctorate	(93.8%)	(87.5%)	(100.0%)	2.13, 0.144
Monthly income
0 to 3400 (USD ~$0.65)	10 (31.3%)	6 (37.5%)	4 (25.0%)
More than 3401 to 5000	22 (68.8%)	10(62.5%)	12 (75.0%)	0.58, 0.446

Staff performance, organizational functioning, and CFIR constructs: bottom 4 x top 4 health facilities

There was a greater proportion of high-performing S&S staff in the top 4 health facilities in comparison to the bottom 4 health facilities, with 3 per cent of high-performing staff in the bottom 4 facilities in comparison to 99 per cent in the top 4 facilities (p = .033). Health care staff burnout was significantly lower in the top 4 health facilities in comparison to the bottom 4 health facilities (M = 39.74 [SE = 2.33] vs. M = 47.39 [SE = 2.19], respectively; p = .025). Of the five CFIR domains (intervention characteristics, outer setting, inner setting, characteristics of individuals, and process), only the process domain was significantly higher in the top 4 health facilities, M = 4.19 (SE = 2.54), in comparison to the bottom 4 health facilities, M = 2.75 (SE = 2.21; p = .005). The process domain constructs, for example, planning, engagement, the presence of internal opinion and champion leaders, the perceived impact of the S&S on stakeholders, intervention fidelity (executing), and reflecting and evaluating were therefore associated with better uptake of the S&S intervention (Table 4).

Table 4.

Staff performance, CFIR domains, and organizational functioning by bottom 4 versus top 4 health facilities

Characteristic	All Mean (SE) n (%)	Bottom 4 Mean (SE) n (%) (n = 16)	Top 4 Mean (SE) n (%) (n = 16)	t, p	Cohen’s d
Staff performance
Proportion of high-performing staff	0.61 (0.22)	0.03 (0.06)	0.99 (0.02)	2.26, 0.033	0.26 (Cramer’s V)
CFIR constructs
Intervention characteristics	1.16 (2.42)	1.31 (2.18)	1.00 (2.71)	1.04, 0.309	0.13
Outer setting	0.25 (1.72)	0.19 (1.64)	0.31 (1.85)	0.08, 0.936	0.07
Inner setting	4.50 (6.13)	4.19 (5.58)	4.81 (6.81)	1.20, 0.241	0.09
Characteristics of individuals	2.19 (2.16)	2.19 (2.74)	2.19 (1.47)	0.58, 0.570	0.00
Process	3.47 (2.45)	2.75 (2.21)	4.19 (2.54)	3.09, 0.005	0.60
Organizational functioning
Provider attitudes	47.00 (6.97)	47.87 (1.79)	48.96 (1.75)	0.44, 0.666	0.27
Barriers to research practice	37.16 (20.95)	34.63 (5.11)	43.14 (5.66)	1.12, 0.275	0.24
Copenhagen burnout inventory	43.47 (8.93)	47.39 (2.19)	39.74 (2.33)	2.40, 0.025	0.46
Organizational readiness	237.25 (31.45)	239.75 (7.73)	239.59 (8.53)	0.01, 0.989	0.16

CFIR Consolidated Framework for Implementation Research.

CFIR process domain, staff burnout, performance, and S&S intervention implementation

To examine the association of CFIR domain, staff burnout, and performance with implementation success, coded qualitative interview data were reviewed by CFIR domain. The average number of constructs identified in the staff interviews was 15 (SD = 4.50).

CFIR process domain

Participants (41%) in the top 4 health facilities reported that planning supplies and equipment (MC kits) needed ahead of time [CFIR construct: Planning], sensitizing the community to the S&S intervention and VMMC before approaching men and women [CFIR construct: Engaging], and inviting them to the health facility to attend S&S sessions and have VMMC performed [CFIR construct: Executing] were associated with implementation success. Improved staff engagement and support by supervisors [CFIR construct: Opinion Leaders, Champions], self-reflection and evaluation and better health facility-level evaluation regarding performance [CFIR construct: Reflecting and Evaluating], and adapting the S&S intervention process and recruitment strategy to suit the needs of the target population also were associated with better staff attitudes toward adopting the S&S intervention [CFIR construct: Executing]. Gradual, systematic strategies to achieve adoption and implementation of the S&S intervention were described as follows:

The members of staff just to continue sensitizing to the community, to the masses so that people get the correct information. – Professional Nurse, Female, 43 years old

We ensured that we had the necessary equipment that we would need to do the services. Now, we have to incorporate the community, show them the services we are providing. They will then recruit more people in the community to come and have the service. – Clinic Officer, Male, 40 years old

We just sit together as a family staff, and then they brief us: we will be having this program going forward at this clinic and it will help. – Counselor, Female, 38 years old

If possible, we can have them do the tests as quickly as possible because when it takes too long, they change their mind about the MC. We follow the clients where they are instead of them coming to us… We should be willing to provide service over the weekend and just give the community a lot of information so they can give the other people information. – Sister in Charge, Female, 44 years old

Staff burnout and performance

All staff reported some degree of dissatisfaction with at least one factor at their facility. Reasons for dissatisfaction included staff, space and supply shortages, conflictive professional relationships, feelings of being unappreciated or unsupported by leaders, and feeling that extra efforts, such as working on weekends, were not rewarded, despite being required to recruit men to undergo VMMC. Staff also reported that moving to different health facilities disrupted job continuity and learning:

We may have a shortage of kits for doing the surgical procedure. – Counselor, Female, 22 years old

You may find that clients are there but maybe there are no providers. That is the big challenge that we may face as for VMMC. – Counselor, Female, 22 years old

As I was saying, with this program, pertaining to these services, it is not so good because we are not supported. We are moved from one place to another. – Clinic Officer, Male, 40 years old

When asked about factors that would help improve the implementation of the S&S intervention, a clinic officer asserted that increasing staff salaries based on performance and job demands as well as addressing supply shortages would help achieve implementation success:

For now they are doing fine, but like I was saying, the payments are not up to date with the performance. So, if they can improve on payments... and we would appreciate the delivery of the sets. – Clinic Officer, Male, 43 years old

Some staff members were perceived to feel oppressed by superiors or that higher-level staff performance differed by expectations across professional hierarchies. Some superiors were perceived as not always being available to see patients or perform VMMCs, despite being trained to do so, which caused the workload to fall on lower-level staff or those untrained to provide services who, as such, could not see patients seeking VMMC services. Some believed that monetary compensation was not commensurate with work effort, availability, and increased workload associated with working on the weekends. Weekend recruitment was viewed as essential, being an ideal time for identifying, educating, and recruiting men for the S&S program, as most men’s work schedules could not accommodate weekday-only operating hours:

Sometimes you may find that there are those that are trained but they are not making themselves available to provide the service. – Counselor, Female, 22 years old

Some staff have been oppressed…. Some are okay but others they feel like they are more superior than others. Staff on the lower side we conduct MCs on weekends and everybody feels that they don’t need to come for work on the weekend. And also when they come for work, the compensation or the allowance is just too low for someone to come from home and then to come and work and then you are given a K40 or K50 (USD $4 to 5) or so. – Professional Nurse, Male, 40 years old

Remedial action

As theorized, identification of factors associated with poor implementation created an “early warning” system, enabling rapid intervention on some elements to enhance intervention uptake. The early warning system was designed to take “snapshots” of the project implementation process to identify sites that were performing suboptimally and initiate rapid remedial action. Using the VMMC and patient attendance data collected on a monthly basis, trends in site performance over a 6 month period were identified following site activation. Qualitative interviews complemented the quantitative assessments and provided additional information related to implementation of the program on site context and readiness to adopt and implement new programs. Applying the coded qualitative information to the CFIR matrix, it was possible to identify the nature of the challenges that arose at the onset of implementation, and to determine whether these issues were affecting performance.

Staff reviewed the challenges identified and related them to the elements of the process of implementation and overall study performance. Issues such as leadership support, time, and attitudes regarding the intervention were reviewed to evaluate their relationship to the performance outcomes, such as recruitment, group leader fidelity to the intervention, and availability of VMMC staff to conduct VMMC. Those issues associated with performance outcomes were addressed by the project coordinator and study and clinic staff, who engaged in a collaborative problem solving process to resolve challenges to implementation at specific sites. As these issues were identified, study staff addressed them at the sites with clinic leadership for remediation. Replacement and/or supportive retraining of low-performing study staff was instituted, both of which resulted in improvement of process elements, that is, staff engagement and support by health care facility, timely supervisors, adaptation of the S&S intervention process and recruitment strategy to better suit the target population, and enhanced staff attitudes toward adoption of the S&S intervention. Senior project administrative staff addressed timely ordering of supplies and equipment, and additional training was undertaken with the coordinating staff, assisting in the process of resensitization of the community advisory board to the S&S intervention and inviting them to attend S&S sessions. At the time of this writing, site performance at low-performing sites was successfully remediated and has been sustained.

DISCUSSION

This study of a VMMC uptake program in Lusaka Province, Zambia, examined potential predictors of implementation success or failure, and identified characteristics of health facilities, providers and staff associated with implementation outcomes. Interim study analyses illustrated that successful intervention implementation required active involvement at individual- and organizational-levels, in championing, planning, executing, and evaluating [5, 18]. As theorized, characteristics associated with poor intervention uptake enabled early identification and remedial action undertaken in the preliminary stages of implementation. In addition, in this study, the process of gradual, systematic adoption, and implementation of the S&S intervention appeared more important than the other CFIR domains, and essential to strengthening the uptake of the program and enhancing sustainability. Finally, staff burnout and transfers to different health facilities posed both individual and structural challenges to implementation, and may contribute to decreased performance and productivity.

Results support the use of the CFIR to guide ongoing intervention implementation and to enhance VMMC program implementation and dissemination in Zambia [5, 18]. Interim analyses highlighted staff performance and attitudes as key components of implementation, and suggest that performance evaluation with remedial feedback was especially important when more frequent coaching is not feasible, as shown in earlier research [10]. In contrast, poor staff performance and clinic staff burnout were linked to low intervention uptake, and support was needed to improve performance and planning, in line with previous work [14]. Increased job demands, and limited resources and managerial support have been previously linked to burnout, and may also affect engagement and performance [21, 22].

This study demonstrates the potential value of the CFIR to maximize uptake of HIV prevention strategies such as VMMC in the Zambian context. The identification and response to challenges in multiple clinics presented an efficient strategy to enhance uptake of VMMC provision in Zambia, an approach that could have an impact on country-wide public health [23]. By consolidating interdisciplinary research on implementation [24], use of the CFIR constructs can enable the development of quality improvement interventions that apply a multifaceted approach to managing systematic challenges to implementation. In an era of diminishing financial support, a CFIR-based “early warning” system could more effectively implement interventions and achieve sustainable clinical services.

Certain limitations must be considered in interpretation and application of these interim findings. The collection of qualitative data and its analysis is a time-consuming process requiring training and expertise. However, despite the constraints of this resource limited setting, qualitative data collection and analysis is a widely used technique in Africa and many African social scientists are trained in its use. As such, investigators should consider the use of the CFIR as an analytic strategy to enhance implementation. The sample size of this study limits generalizability to other populations and statistical power to detect differences between unsuccessful and successful health facilities. However, as a planned interim analysis to identify contributors to implementation success early within a 5 year dissemination and implementation program, the smaller sample was useful in guiding ongoing implementation and scale-up of the S&S intervention.

This study illustrates the potential utility of a dissemination and implementation model such as the CFIR to guide program decision making in the Zambian context. Early identification of challenges to implementation may create a pathway for remedial action to enhance the likelihood of program sustainability. HIV prevention interventions require effective monitoring to enhance the potential to achieve dissemination, implementation, and sustainability goals and bridge the gap between research and practice [25, 26].

Compliance with Ethical Standards

Conflicts of Interest: The authors declare no potential conflict of interest.

Primary Data: The findings reported have not been previously reported, published, and the manuscript is not being simultaneously submitted elsewhere. The authors have full control of all primary data and agree to allow the journal to review their data if requested.

Authors’ Contributions: S. M. Weiss, D. L. Jones, R. Zulu, and N. Chitalu conceived the study. V. J. Rodriguez drafted the manuscript with S. A. Butts. V. J. Rodriguez conducted all analyses with oversight by K. Arheart and S. M. Weiss. S. M. Weiss, D. L. Jones, R. Zulu, and N. Chitalu participated in study execution and coordination. S. M. Weiss, D. L. Jones, R. Zulu, and N. Chitalu assisted in editing and finalizing the manuscript and gave critical review. All authors read and approved the final manuscript.

Ethical Approval: All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. No animals were used in this study.

Informed consent: Informed consent was obtained from all participants prior to participating in the study.