Champion and audit and feedback strategy fidelity and their relationship to depression intervention fidelity: A mixed method study

Abstract

Background:

Globally, mental health disorders rank as the greatest cause of disability. Low and middle-income countries (LMICs) hold a disproportionate share of the mental health burden, especially as it pertains to depression. Depression is highly prevalent among those with non-communicable diseases (NCDs), creating a barrier to successful treatment. While some treatments have proven efficacy in LMIC settings, wide dissemination is challenged by multiple factors, leading researchers to call for implementation strategies to overcome barriers to care provision. However, implementation strategies are often not well defined or documented, challenging the interpretation of study results and the uptake and replication of strategies in practice settings. Assessing implementation strategy fidelity (ISF), or the extent to which a strategy was implemented as designed, overcomes these challenges. This study assessed fidelity of two implementation strategies (a ‘basic’ champion strategy and an ‘enhanced’ champion + audit and feedback strategy) to improve the integration of a depression intervention, measurement based care (MBC), at 10 NCD clinics in Malawi. The primary goal of this study was to assess the relationship between the implementation strategies and MBC fidelity using a mixed methods approach.

Methods:

We developed a theory-informed mixed methods fidelity assessment that first combined an implementation strategy specification technique with a fidelity framework. We then created corresponding fidelity indicators to strategy components. Clinical process data and one-on-one in-depth interviews with 45 staff members at 6 clinics were utilized as data sources. Our final analysis used descriptive statistics, reflexive-thematic analysis (RTA), data merging, and triangulation to examine the relationship between ISF and MBC intervention fidelity.

Results:

Our mixed methods analysis revealed how ISF may moderate the relationship between the strategies and MBC fidelity. Leadership engagement and implementation climate were critical for clinics to overcome implementation barriers and preserve implementation strategy and MBC fidelity. Descriptive statistics determined champion strategy fidelity to range from 61 to 93% across the 10 clinics. Fidelity to the audit and feedback strategy ranged from 82 to 91% across the 5 clinics assigned to that condition. MBC fidelity ranged from 54 to 95% across all clinics. Although correlations between ISF and MBC fidelity were not statistically significant due to the sample of 10 clinics, associations were in the expected direction and of moderate effect size. A coefficient for shared depression screening among clinicians had greater face validity compared to depression screening coverage and functioned as a proximal indicator of implementation strategy success.

Conclusion:

Fidelity to the basic and enhanced strategies varied by site and were influenced by leadership engagement and implementation climate. Champion strategies may benefit from the addition of leadership strategies to help address implementation barriers outside the purview of champions. ISF may moderate the relationship between strategies and implementation outcomes.

1. Introduction

Globally, non-communicable diseases (NCDs) account for 16 million premature deaths under the age of 70 and 82% occur in low and middle-income countries (LMICs) (WHO, 2014). The co-occurrence of NCDs and depression results in worse health outcomes compared to either condition alone. In their analysis of the WHO World Health Survey, Moussavi et al. (2007) concluded that those with co-occurring depression and one or more chronic diseases constituted the most severe outcomes of all disease states (Moussavi et al., 2007). Mental health and substance abuse disorders themselves rank as the foremost causes of disability and LMICs hold a disproportionate share of the global burden (Institute for Health Metrics and Evaluation, 2017; Whiteford et al., 2013). Depression alone is responsible for 50 million years lived with disability (YLD) annually and more than 80% of this burden occurs in LMICs (WHO, 2017a). The high prevalence of depression in LMICs and the severity when co-occurrent with NCDs, has resulted in calls to scale up integrated approaches to treating mental health disorders within NCD care (Patel and Chatterji, 2015). However, while treatments for mental health disorders and NCDs have been effective in LMIC settings, they are rarely implemented due to a host of barriers (Thornicroft, 2012; Mendis et al., 2012). Researchers have described the need for implementation strategies to address these barriers to facilitate widespread and effective implementation of evidence-based interventions (Thornicroft, 2012; Wainberg et al., 2020; Betancourt and Fazel, 2018; Murray et al., 2014; Keynejad et al., 2018).

Malawi is a central sub-Saharan African country of 19.7 million with an economy designated as low-income (WBO, 2022). The prevalence of depression and other common mental disorders range between 10.7 and 30.4% compared to just 4.4% globally, and between ~6 and 7% within the African region (Stewart et al., 2014; Stewart et al., 2010; Udedi, 2014; WHO, 2017b). Regarding suicide, the global age-standardized suicide rate is 9.0 per 100,000 population, the African region is the highest among all regions at 11.2 with Malawi standing at 10.6 (World Health Organization, 2021). Despite the high burden of depression and suicide, a lack of trained psychiatric specialists, the need to serve a largely rural population, and the structural organization of Malawi’s healthcare system, combine to pose significant challenges to treatment provision. With less than 1 psychiatrist and 2.5 psychiatric nurses per 100,000 population, mental health treatment is rarely integrated into primary or secondary care (Jacob et al., 2007). Most patients with mental health disorders are only able to access free care at one of 4 public central hospitals located in urban areas, posing barriers to service accessibility (Udedi, 2016; Ahrens et al., 2020). At these hospitals, services are focused mostly on pharmacological and physical treatments for those experiencing severe mental health disorders (Udedi, 2016; Ahrens et al., 2020; Zimba et al., 2021). Mental health treatment, organized in this way, makes resources available mostly for those experience severe mental health disorders in urban areas, creating a gap in service across lower levels of the healthcare system, potentially leaving individuals with less severe mental health disorders underdiagnosed and undertreated (Zimba et al., 2021).

Given the challenge of limited psychiatric specialists, many efforts to respond to the burden of mental health disorders in LMICs and in Malawi have focused on task-shifting strategies to expand treatment capacity. Task-shifting is an implementation strategy focused on training non-specialists to deliver specialized services like prescribing antidepressant medication or providing psychosocial counseling (Galvin and Byansi, 2020; Johns et al., 2018). In Malawi, task shifting efforts have proven feasible in integrating depression screening, psychosocial counseling, and anti-depressant medication prescription into routine HIV and perinatal care, but have been further hindered by lack of intervention fidelity and sustainability (Kulisewa et al., 2022; Stockton et al., 2020; Udedi et al., 2018).

This manuscript represents a secondary analysis within a parent trial (Gaynes et al., 2020). The parent trial looked to build upon the feasibility of task shifting models and respond to challenges therein by providing additional implementation strategies to support healthcare providers: champion and audit and feedback strategies, described further in our methods section.

Implementation strategies are defined as “methods or techniques used to enhance the adoption, implementation, and sustainability of a clinical program or practice,” with the ultimate goal of improving clinical outcomes at the patient level (Proctor et al., 2013). However, implementation strategies are often complex, encompassing a multitude of components and mechanisms that are not well defined or documented (Bradley et al., 1999; Lewis et al., 2020; Nadeem et al., 2013). This lack of clarity contributes to a vague ‘black box’ of activities that hamper the uptake and replication of implementation strategies in practice settings (Hulscher et al., 2003). Implementation strategy fidelity (ISF) is defined as carrying out the strategy as it was designed (Slaughter et al., 2015). Fidelity assessment facilitates the evaluation of a Type-III research error: failure to implement a strategy as planned, leading to an erroneous conclusion that null results are due to attributes of the strategy itself (Slaughter et al., 2015; Dusenbury et al., 2003a; Gearing et al., 2011; Summerfelt, 2003; Dobson and Cook, 1980). Fidelity also operates as a moderator of main effects pathways where efficacious interventions carried out with higher fidelity tend to yield better clinical outcomes compared to the same interventions delivered with lower fidelity (Dusenbury et al., 2003a; Dane and Schneider, 1998; Durlak and DuPre, 2008). However, this relationship has yet to be carefully examined at the level of the implementation strategy. Implementation researchers focused on global mental health have described the need to better understand implementation strategies, and their fidelity, to ultimately improve their optimization and scale-up (Wainberg et al., 2020; Saxena, 2016; Akiba et al., 2021).

Given the gaps related to ISF assessment and its association with study outcomes, the present study’s goal focused on (WHO, 2014) assessing fidelity of the parent study’s implementation strategies; (Moussavi et al., 2007) assessing fidelity of the depression intervention whose integration the implementation strategies were intended to support; and (Institute for Health Metrics and Evaluation, 2017) exploring the relationship between the two using a mixed methods approach.

2. Methods

2.1. Study setting

Malawi provides most of its healthcare through government-run health facilities (Masefield et al., 2020). Healthcare administration is generally organized through a hierarchy where the Ministry of Health (MoH) oversees District Health Management Teams (DHMTs) responsible for healthcare delivery throughout Malawi’s 28 district hospitals (Bulthuis et al., 2020). Management of NCDs in Malawi is largely provided through NCD clinics within district hospitals that specialize in treating hypertension, diabetes mellitus, asthma, and epilepsy (Manjomo et al., 2016). Within each NCD clinic, the DHMT nominates one provider to act as an NCD Coordinator – tasked with organizing the day-to-day NCD clinic operations (Zimba et al., 2021). While prior research has focused on integrating mental health services into routine HIV and primary care in Malawi (Stockton et al., 2020; Wright et al., 2013), mental health services had not been integrated within NCD care at the time of the parent trial.

2.2. Parent trial

This analysis uses data from the Sub-Saharan Africa Regional Partnership (SHARP) for Mental Health Capacity Building trial (NIMH [U19MH113202, 2017]; ClinicalTrials.gov ID NCT03711786). SHARP is a clinic-randomized implementation trial comparing the success of a basic vs. an enhanced implementation strategy in achieving the integration of evidence-based depression treatment into routine care at 10 NCD clinics across Malawi (Gaynes et al., 2020). Five clinics were assigned to the enhanced strategy (clinic #s 1-5) and five to the basic strategy (clinic #s 6-10) using constrained randomization. The study’s active intervention period ran from November 2019–September 2021 (henceforth “the study period”).

2.3. Depression treatments

The parent trial used measurement based care (MBC) through a task shifting approach. Non-psychiatric practitioners were trained to assess depression using the Patient Health Questionnaire (PHQ-9) and treat identified cases with antidepressant medication (amitriptyline, fluoxetine) determined by a treatment algorithm (Adams et al., 2012). MBC also encompasses an assessment of suicidal ideation paired with as needed suicide risk assessments (SRA) and additional treatments. After patients began antidepressant medication, follow-up assessments were scheduled every 4 weeks. During each follow-up, practitioners readministered the PHQ-9, assessed for negative side-effects, and adjusted treatment intensity in-line with the algorithm. For most patients undergoing treatment in MBC, depression remission is achieved after 12 weeks (Adams et al., 2012). MBC has proven effectiveness in the context of randomized and quasi-experimental studies in the US and China (Trivedi et al., 2007; Guo et al., 2015).

The parent trial packaged MBC alongside Friendship Bench (FB) psychosocial counseling to provide patients with an array of depression treatment services. Developed in Zimbabwe, FB uses problem-solving therapy via one-on-one counseling to address symptoms of common mental disorders (Chibanda et al., 2015). The present study focuses only on the assessment of MBC fidelity given the implementation strategies’ focus on integrating it within routine NCD services. FB training and supervision were mostly addressed through FB experts external to the implementation strategies.

2.4. Basic and enhanced implementation strategies

The basic implementation strategy employed a champion model. Champions, defined as individuals who work diligently and persistently within organizations to facilitate intervention implementation, have been found feasible and effective in a variety of clinical settings (Miech et al., 2018; Shea, 2021). Hospital leadership at each of the parent study’s ten sites nominated one champion and two alternates (to protect against attrition). Champion duties included: (WHO, 2014) training NCD providers in MBC (both formal and on-the-job training); (Moussavi et al., 2007) staffing each NCD clinic day with NCD providers trained in MBC (NCD clinics operated between 1 and 3 days/week); (Institute for Health Metrics and Evaluation, 2017) leading supervision of NCD providers trained in depression care; (Whiteford et al., 2013) ensuring accurate recordkeeping and tracking the quality of depression care integration through monthly reporting; (WHO, 2017a) ensuring stable antidepressant medication supply; and (Patel and Chatterji, 2015) coordinating clinical care with the FB counselors. Research staff trained these champions in MBC and their champion job role at the outset of the parent study.

The enhanced strategy included everything in the basic strategy plus quarterly supportive supervision with audit and feedback from external supervisors. Audit and feedback is a strategy that prompts improvements in intervention implementation by systematically providing performance evaluations to implementors (often medical providers) (Ivers et al., 2012). Ideally, the strategy takes place over multiple “audit cycles” where auditors assess clinical implementation and provide feedback at regular and meaningful intervals (Flottorp et al., 2010). Ivers et al. (2014) meta-regression of results from audit and feedback trials concluded that feedback worked best when “delivered by a supervisor or respected colleague; presented frequently; featuring both specific goals and action-plans; aimed to decrease the targeted behavior; in settings where baseline performance was lower; and recipients were non-physicians (Ivers et al., 2014).” In mental healthcare settings, completing “audit cycles” may be most successful when supported by a facilitator or individual responsible for assessing the implementation process (Pedersen et al., 2018). In the parent trial, an audit team consisting of NCD, MBC, and FB experts from the Malawi MoH and study team comprised the audit team that visited the five sites in the enhanced condition. Each visit lasted 2–3 days and involved (WHO, 2014) an assessment of depression care records; (Moussavi et al., 2007) clinical observation of NCD providers and FB counselors; (Institute for Health Metrics and Evaluation, 2017) interviews with NCD providers and FB counselors; (Whiteford et al., 2013) a feedback session with NCD providers and hospital leaders focused on challenges identified through the audit paired with suggestions on how to overcome them; and (WHO, 2017a) a written report submitted to the hospital leaders at each clinic outlining the audit findings and recommendations for improvements.

2.5. Fidelity framework

Our approach to fidelity assessment at the intervention and implementation strategy levels utilized Carroll et al.’s conceptual framework for implementation fidelity, which organizes fidelity into four domains of content (overall delivery of core components), coverage (proportion of content delivered), frequency (of content delivery), and duration (of delivered content) (Carroll et al., 2007). The level of these fidelity components may also be impacted by three elements (WHO, 2014) comprehensiveness of strategy description (Moussavi et al., 2007), quality of delivery, and (Institute for Health Metrics and Evaluation, 2017) participant responsiveness (Carroll et al., 2007). The comprehensiveness of strategy description pertains to how well an intervention or strategy’s components are defined (Carroll et al., 2007). Quality is defined as the way an actor delivers an intervention or strategy (Carroll et al., 2007; Mihalic, 2004). Participant responsiveness refers to how well action targets are engaged by an intervention or strategy (Carroll et al., 2007).

Our first step in assessing basic and enhanced ISF began with specifying each strategy. Implementation strategy specification facilitates strategy component identification, measurement, and replicability by defining the specifics of each component (Proctor et al., 2013; Pinnock et al., 2015; Hoffmann, Glasziou, Boutron, Milne, Perera, Moher et al.). Accordingly, we adopted the Proctor et al. (2013) approach to specify the basic and enhanced strategies (Table 1). Given our focus on fidelity assessment and the conceptual overlap with Proctor et al. (2013)’s inclusion of dose into their specification model, we expanded the dose classification to include additional fidelity components consistent with fidelity measurement (Carroll et al., 2007). Table 1‘s fidelity description column details the conceptual ideal for each component’s delivery, defined by the study team and stakeholders at the Malawi MoH.

Table 1.

Basic and enhanced strategy specification with expanded fidelity.

Table 1. Champion strategy (Basic) and Champion + Audit and feedback strategy (Enhanced) Specification
Strategy Name	Definition	Actor	Action	Action Target	Temporality	Fidelity Descriptiona	Fidelity Assessmenta • Fidelity domain: Assessment (Data source)	Implementation Outcome Affected	Justification
Basic strategy (champion)	Initial, Refresher, and On-the-job trainings	Champion	Trains fellow clinicians who provide NCD services	Providers of NCD services	Initial training, Study year 1 Ongoing-as-needed training, Study years 2, 3 and 4	• Content: MBC intervention • Coverage: All NCD providers • Freq: 1 initial didactic training, 2 didactic refresher trainings, on-the-job training as needed • Duration: 8hrs/didactic training, on-the-job flexible	• Content: Delivery of training materials (Pre-post training test scores) • Coverage: Proportion of NCD providers trained in MBC (Staffing logs) • Freq: # of trainings (Study budget/training logs) • Duration: Not assessed quantitatively • Quality and participant responsiveness: (Qualitative interviews)	Fidelity, cost, sustainability	Training is an effective means of increasing EBI knowledge and skills in clinicians in LMICs and may increase the sustainability of EBIs
	Clinic coverage with trained providers	Champion	Maintains schedule of trained providers such that all patients receive MBC	Providers trained in MBC	Ongoing, Study years 1-4	• Coverage: All NCD providers trained in MBC	• Coverage: Proportion of NCD providers on duty trained in MBC (Clinic duty logs) • Quality and participant responsiveness: (Qualitative interviews)	Penetration	Clinicians in middle-management positions can increase the diffusion of team based EBIs
	Provide clinical supervision	Champion	Supervises providers trained to deliver MBC	Providers trained in MBC	Ongoing, Study years 1-4	• Content: MBC intervention • Coverage: All NCD providers • Freq: 1/month	Not assessed quantitatively	Fidelity	Clinical supervision is an effective means of ensuring clinical EBI adherence. Supervision also decreases burnout and turnover among those implementing the EBI
	MBC Reporting	Research team + Champion	Research team provides MBC specific forms and Qualtrics report of integration indicators for champion to complete monthly	Champion	Initial and ongoing provision of MBC forms, Study years 1-4 Ongoing reporting, Study years 1-4	• Freq: 1/month	• Freq: Timeliness of monthly submissions (MBC report) • Quality and participant responsiveness: (Qualitative interviews)	Feasibility	The design and provision of MBC records increase the feasibility of record keeping and reporting on behalf of the champion
	Ensuring anti-depressant medication supply	Champion	Liaises with Central Medical Stores to ensure consistent and adequate supply of anti-depressant medications	Anti-depressant medication stock	Ongoing, study years 1-4	• Coverage: Adequate medication doses/patient population • Freq: 1 order or each medication/month	• Coverage: Both AMI and FLU ordered (MBC report) • Freq: 1 order of each medication/month (MBC report) • Quality and participant responsiveness: (Qualitative interviews)	Fidelity	Anti-depressant medications constitute the core component of MBC
	Coordinating clinical care with FB counselors	Champion	Ensures working conditions that facilitate patient referral and treatment information between NCD providers and FB counselors	Providers trained in MBC and FB counselors	Ongoing, Study years 1-4	• Content: Successful FB referral • Coverage: All patients referred to FB counseling • Freq: not applicable • Duration: not applicable	Not assessed quantitatively	Penetration	Clinicians in middle-management positions can increase the diffusion of team based EBIs
Enhanced strategy (champion + audit and feedback)	Audit andprovide feedback	Audit team (MBC expert(s), FB expert(s), Ministry of Health representative(s))	Provides ongoing on-site evaluation of champion activities and provides in-person and written feedback that reinforces successes, identifies challenges, suggests solutions to overcome identified challenges	Champion, Hospital leadership, FB supervisor	Ongoing quarterly audits, Study years 1-4	•Content: Clinic observation, interviews, and record reviews •Coverage: Presence of audit team throughout visit •Freq: not applicable •Duration: 2–3 days, 30+ min for opening and closing meetings	•Content: Audit and feedback completion (Audit form) •Coverage: Presence of audit team throughout visit (Audit form) •Duration: 2–3 days, 30+ min for opening and closing meetings (Audit form) •Quality and participant responsiveness: (Qualitative interviews)	Fidelity	Audit and feedback can improve EBI fidelity in healthcare settings

^aFidelity expands the original “dose” component to include additional fidelity indicators consistent with Carroll et al. (2007).

We applied this fidelity framework to the study’s implementation strategies after study launch and therefore not every fidelity domain or strategy component were included in the study’s data collection plan. Table 1‘s fidelity assessments column describes our operationalization of fidelity assessment to each corresponding fidelity domain paired with the best available data collection modalities and data sources.

Below, we describe the assessment of MBC fidelity as well as fidelity assessment of each basic and enhanced strategy component.

2.6. Intervention fidelity: MBC

We conceptualized MBC fidelity in two parts: screening coverage, and algorithm content adherence. We assessed screening coverage by calculating the proportion of NCD patients screened for depression using the PHQ-9 during the study period (Lewis et al., 2015). Algorithm content adherence was defined as the proportion of patients with (WHO, 2014) PHQ-9 scores ≥ 10 who initiated an antidepressant at an appropriate dose (Moussavi et al., 2007), PHQ-9 scores between 5 and 9 who either initiated an antidepressant at an appropriate dose and/or were referred to FB counseling. We included an additional content fidelity indicator for SRA administration that assessed (Institute for Health Metrics and Evaluation, 2017) the proportion of patients evaluated by providers using the SRA who expressed suicidal ideation. Due to inconsistent clinical documentation of clinical actions at return visits, we focused our measurement of content fidelity on providers’ initial treatment visit with each patient identified with depression. The two fidelity components (coverage and content) yielded percentage scores, which we averaged at the clinic level to assess total MBC fidelity.

2.7. Implementation strategy fidelity (ISF)

In this subsection we describe each strategies’ components. Table 1‘s fidelity “description” and “assessment” columns provide additional information regarding fidelity domain specific data sources and assessments. We assessed fidelity to the champion (all 10 clinics) and audit and feedback (5 clinics randomized to the enhanced arm) strategies separately. Fidelity to the champion strategy represents each champion’s fidelity while audit and feedback fidelity represents the audit team’s fidelity. Due to this distinction, we do not combine champion and audit and feedback fidelity scores for sites in the enhanced condition. Instead, we report champion and audit and feedback fidelity scores side-by-side at the clinic level.

Basic and enhanced - Initial, refresher, and on-the-job training: Fidelity to this component focused most intently on formal and documented initial and refresher trainings. We found on-the-job training more difficult to assess given its provision on an as needed basis often occurring during the clinic day and lacking accompanying documentation.

Basic and enhanced - Clinic coverage with MBC trained providers: After training a cadre of providers on MBC, champions were tasked with scheduling trained providers to work on every clinic day so that all NCD patients could benefit from depression screening and potential treatment.

Basic and enhanced - Providing MBC supervision: The champion role also focused on monthly provision of clinical supervision to improve provider MBC adherence. While the component featured as part of champions’ initial training, structured supervision was seldom put into practice by the champions due to multiple competing demands. When champions did hold supervision, formats varied widely from brief checkins during clinic service to more structured review of paper records. The lack of systematic supervision left study staff without the necessary information to adequately collect supervision data.

Basic and enhanced - MBC Reporting: Champions submitted a monthly report of depression indicators to the study team and the MoH. Reports were submitted via an online survey created by the research team. Champions were provided a small monthly mobile data bundle (~$14 USD) from the study team to facilitate online submission.

Basic and enhanced - Ensuring anti-depressant medication supply: MBC integration required consistent access to amitriptyline (AMI) and fluoxetine (FLU). While both medications were available to NCD clinics, they were not regularly stocked prior to MBC integration. Champions were tasked with tracking their clinic’s supply of each medication monthly and ordering new doses as necessary to avoid stock outs.

Basic and enhanced - Coordinating clinical care with FB counselors: FB counseling often took place outside the NCD clinic, commonly at an adjacent structure or outdoor space within the same hospital. This created a need for an effective patient handoff between NCD providers and FB counselors. Once patients began FB counseling, there was also a need for FB counselors to communicate with NCD providers about patients’ depression treatment. Ensuring effective patient referrals and coordinating clinical care between the NCD providers and FB counselors fell under the champion’s role description. However, like the supervision component, the manner in which champions coordinated with FB counselors varied across study sites and was therefore difficult to assess quantitatively. Instead, the quality of this component was assessed during our interviews.

Enhanced only - Audit and feedback: Audit and feedback visits for sites in the enhanced condition took place every 3–4 months, comprising 7 visits in total during the study period. While originally intended to improve MBC fidelity, audit and feedback visits also focused on improving adherence to the champion strategy, a relationship explored further in the results section.

2.8. Qualitative data collection

Qualitative interviews were conducted for the parent trial at only 6 of the 10 study clinics (clinics 1, 4, 5, 6, 8, and 9) due to study resource limitations. However, a purposive clinic sampling strategy targeted information rich clinics that varied on three factors relevant to implementation: anecdotal ISF as assessed by study staff (3 higher and 3 lower performing sites), implementation strategy condition (3 basic and 3 enhanced), and geography (2 sites each from Malawi’s Northern, Central, and Southern Regions) (Palinkas et al., 2015). Seven participants were recruited from each of these 6 clinics (1 champion, 2 NCD providers, 1 FB counselor, 1 hospital leader, 1 patient, and 1 clinic-embedded research staff member) as well as 3 audit team members (n = 45). Interviews focused on the delivery of the implementation strategies, particularly exploring the domains of quality, participant responsiveness, and the comprehensiveness of the strategy’s descriptions. Interviews also explored the impact of ISF on MBC fidelity. Interviews took place in English except for FB counselor and patient interviews which were done in Chichewa and Tumbuka. Those interviews were translated and transcribed into English in one step by a bi-lingual transcriptionist due to parent study budget and time constraints.

2.9. Mixed methods approach

We adopted a mixed methods approach to capitalize on the volume of rich qualitative and quantitative data collected through the parent study. Palinkas et al. (2011) describe how mixed methods approaches should attend to three criteria: structure, function, and process (Palinkas et al., 2010). The parent study’s structure gave equal weighting to quantitative and qualitative data with simultaneous data collection. Our goal for this study was to use both types of data to fill in the gaps when assessing ISF and exploring its relationship with MBC fidelity. To that end, we utilized convergence by using “both types of methods to answer the same question, through comparison of results to see if they reach the same conclusion (triangulation) (Palinkas et al., 2010).” We also utilized the process of data merging, where qualitative and quantitative datasets were combined, facilitating more direct comparison. Qualitative data were analyzed using Reflexive Thematic Analysis (RTA), a qualitative method utilizing a flexible inductive and deductive approach to identify patterns of meaning in a dataset (Braun and Clarke, 2006; Braun et al.,2019). Quantitative analysis focused on descriptive statistics (means,standard deviations, ranges, and correlations) and comparing variables visually through scatterplots. Hypothesis testing was not appropriate due to our small sample size of 10 clinics (Shieh, 2006). Nevertheless, Pearson correlation coefficients are presented alongside their 95% confidence intervals to give readers a sense of the quantitative relationship among continuous fidelity scores. While we carried out qualitative and quantitative analyses concurrently, some analytical processes informed one another in a sequential manner. The subsection below describes how a quantitative indicator was developed to assess a phenomenon described by participants during their qualitative interviews. Both concurrent and sequential analyses helped build our understanding of key study variables and themes presented in the results section.

2.10. Shared screening coefficient

We developed a metric based on the Gini coefficient, typically utilized to assess society level inequality, labeled the shared screening coefficient (Gastwirth, 1972). Our mixed-methods analysis illuminated disparity among each clinic’s NCD providers regarding the proportion of patients they screened for depression. To assess the extent of this disparity, we created the shared screening coefficient to assess the extent to which patient depression screening was equally shared among clinic providers. A coefficient of 1 indicates perfectly shared screening with all providers screening an equal number of patients, whereas a coefficient of 0 indicates perfectly unequal screening with one provider responsible for all screened patients. While not an indicator of fidelity, the shared screening coefficient may represent a proximal indication of implementation strategy effectiveness.

2.11. Ethical approval

The parent trial was approved by Malawi’s National Health Sciences Research Committee (NHSRC Protocol# 2143) and by the Office of Human Research Ethics at the University of North Carolina-Chapel Hill (IRB# 18-2211). The present study was approved by the Office of Human Research Ethics at the University of North Carolina-Chapel Hill (IRB# 20-3705).

3. Results

We first describe the quantitative fidelity scores for both implementation strategies and the MBC intervention before describing correlations focused on the relationship between them. We then present key qualitative themes regarding inter-relationships between implementation barriers, leadership, and implementation climate as well as their impact on ISF and MBC fidelity. This section ends with a description of the shared screening coefficient, a proximal indicator of implementation strategy effectiveness developed through our mixed methods process,and its relationship with both ISF and MBC fidelity.

3.1. Basic and enhanced fidelity and MBC fidelity

Overall fidelity to the champion strategy components ranged from 61 to 93% across the 10 clinics ($\overline{\mathrm{x}}=82\%$, sd = 9%) (Table 2). Initial, refresher, and on-the-job training fidelity averaged 89% and ranged from 67 to 99% across all sites (sd = 10%). Clinic coverage fidelity ranged from 51 to 100% ($\overline{\mathrm{x}}=81\%$, sd = 12%). Fidelity to ensuring antidepressant medication supply averaged 81% and ranged from 52 to 97% across sites (sd = 13%). Reporting fidelity ranged from 38 to 98% ($\overline{\mathrm{x}}=76\%$, sd = 17%). Fidelity to the audit and feedback strategy ranged from 82 to 91% ($\overline{\mathrm{x}}=88\%$, sd = 3%) for audit teams visiting the 5 clinics assigned to the enhanced condition.

Table 2.

Basic, enhanced, and MBC fidelity.

Table 2. Champion (Basic), Champion + Audit and Feedback (Enhanced), and MBC Fidelity
Clinic	Workload (Avg. patients/month)	Initial, Refresher, and On-the-job Training				Clinic Coverage	Ensuring AD Medication Supply			Reporting	Champion Fidelity	Audit and Feedback				MBC Fidelity
		Con.	Cov.	Freq.	Total.	Cov.	Cov.	Freq.	Total	Freq.		Con.	Cov.	Dur.	Total	Con.	Cov.	Total
1	262	89%	98%	100%	96%	81%	92%	100%	96%	98%	93%	92%	96%	81%	90%	91%	100%	95%
2	198	94%	97%	100%	97%	84%	90%	88%	89%	90%	90%	95%	98%	81%	91%	76%	74%	75%
3	287	97%	100%	100%	99%	89%	92%	58%	75%	98%	90%	90%	89%	67%	82%	80%	73%	77%
4	856	90%	96%	100%	95%	83%	100%	60%	80%	71%	82%	93%	96%	76%	88%	99%	26%	62%
5	152	62%	98%	100%	87%	89%	50%	98%	74%	60%	77%	91%	94%	76%	87%	77%	45%	61%
6	213	90%	100%	100%	97%	100%	97%	97%	97%	70%	91%	–	–	–	–	93%	97%	95%
7	221	80%	70%	100%	83%	69%	92%	89%	91%	70%	78%	–	–	–	–	93%	15%	54%
8	321	86%	83%	33%	67%	82%	100%	60%	80%	76%	76%	–	–	–	–	92%	67%	80%
9	548	88%	86%	100%	91%	81%	83%	21%	52%	84%	77%	–	–	–	–	88%	63%	76%
10	215	88%	100%	33%	74%	51%	60%	100%	80%	38%	61%	–	–	–	–	88%	71%	80%
$\overline{\mathrm{x}}$	327	86%	93%	87%	89%	81%	86%	77%	81%	76%	82%	92%	95%	76%	88%	88%	63%	75%
Sd	205	9%	9%	27%	10%	12%	16%	25%	13%	17%	9%	2%	3%	5%	3%	7%	26%	13%

Fidelity to the MBC intervention also varied across sites (54–95%, $\overline{\mathrm{x}}=75\%$, sd = 13%). Disaggregating into fidelity domains, MBC coverage ranged from 15 to 100% ($\overline{\mathrm{x}}=63\%$, sd = 26%) and MBC content ranged from 76 to 99% ($\overline{\mathrm{x}}=88\%$, sd = 7%).

To examine the relationship between champion strategy fidelity and MBC fidelity, our initial analysis focused on the champion components that might directly impact MBC fidelity: training, clinic coverage, and ensuring stable anti-depressant medication supply. These champion strategy components shared a positive correlation with MBC fidelity, but the correlation was not statistically significant (r = .38, 95% CI [−0.26,0.78]) (Table 3). We then examined the relationship between the champion components of training and clinic coverage with MBC coverage, theorizing those components to more directly impact the extent to which providers screened patients for depression, we found a weaker but still moderately positive correlation that was also not statistically significant (r = .26, 95% CI [−0.20,0.41]) (Table 3).

Table 3.

Key variable correlations.

x Component(s)	y Component(s)	r	95% CI Low	95% CI High
Champion fidelity (training, clinic coverage, ensuring AD medication supply)	MBC fidelity (coverage, content)	0.38	−0.26	0.78
Champion fidelity (training, clinic coverage)	MBC fidelity (coverage)	0.26	−0.20	0.41
Champion fidelity (training, clinic coverage, ensuring AD medication supply, reporting)	Shared screening coefficient	0.58	−0.06	0.89
Shared screening coefficient	MBC fidelity (coverage)	0.41	−0.16	0.55

3.2. Leadership engagement, implementation climate, and barriers to implementation strategy and MBC fidelity

RTA illuminated key determinants of ISF and MBC fidelity, several themes focused on barriers that impacted ISF and MBC fidelity, and the role of hospital leadership engagement and implementation climate in addressing implementation barriers. Nearly every participant described similar barriers threatening fidelity of the champion strategy and the MBC intervention. Barriers included provider resistance to MBC, high workloads due to MBC integration, provider turnover, the COVID-19 pandemic, and clinic environments not conducive to MBC. Despite identifying common barriers at all clinics, participants’ description of implementation climate and leadership engagement varied notably across clinics. Participants described the role of implementation climate within their clinics, noting how barriers were more easily overcome when new strategies and interventions were better rewarded, supported, and expected. Respondents in some clinics described how engaged leadership and strong implementation climates helped to better ensure champion strategy fidelity and MBC fidelity by overcoming implementation barriers.

Most respondents described how hospital leaders, specifically known as DHOs and DMOs, wielded power to overcome contextual barriers and ultimately better ensure fidelity of the champion strategy and MBC intervention. A District Health Officer (DHO) represents the highest position within any district. Working from the district hospital, the DHO is responsible for healthcare provided within their respective district. The District Medical Officer (DMO) is responsible for ensuring day-to-day clinical care at the district hospital, reporting to the DHO and directly supervising the champion. Our analyses determined a range of leadership engagement across the 6 clinics included in the qualitative sample. More engaged leaders were able to name all or most of the champion strategy’s components, tended to be present at the NCD clinic, described equal prioritization of depression and NCD service provision, shared an awareness of implementation barriers, and facilitated efforts to overcome them. Participants at Clinics 1, 6, and 4 described this more engaged style of leadership; overall champion fidelity scores at those sites were 93, 91, and 82% respectively, all at or above the study average of 82% (Table 2). Less engaged leadership was described at Clinics 5, 8, and 9 where overall champion fidelity scores were below the study average at 77, 76, and 77% respectively (Table 2). The champion from Clinic 1, which had the highest champion strategy fidelity score (93%) and shared the highest MBC fidelity score (95%, along with Clinic 6) (Table 2), described their relationship with their hospital’s DMO, and the DMO’s ability to address implementation barriers:

[My relationship with the DMO is] very good because we are able to interact on issues that are concerning … I would say generally [the DMO] is very much interested in the things that are happening at the clinic. So, when we have problems at the clinic, [They] are able to come in time. I can discuss issues pertaining to the clinic or the depression screening with [the DMO]. I can talk to [the DMO] freely through WhatsApp, through a call … I would say [the DMO] is very much interested in all the activities that are happening at the clinic in general. – Clinic 1, Champion

This champion’s description included the DMO’s interest in the champion strategy and MBC intervention, the champion’s ability to freely discuss issues through multiple communication channels, and the DMO’s willingness to intervene regarding problems that impacted depression screening. When the DMO referenced in the quote above was asked about their relationship with the champion, they said:

If there are challenges in the NCD clinic that need my attention, [the champion] does report to me and we work together to solve that; be it whether the room they stay … or anything that [the champion] feels … So to talk of the relationship, we work hand in hand to address such [issues]. – Clinic 1, DMO

This level of leadership support seemed to exist within a clinic environment where the champion strategy and MBC were well supported and rewarded. The same Clinic 1 champion relayed that:

When you build a good relationship with yourself and the supervisors, the clinicians, and the nurses, I find that the work is very easy because now they take the program as their own. They could go to the clinic even before you call them to go to the clinic … so I feel like [when] they have the responsibilities, they take it as their responsibility. – Clinic 1, Champion

The Clinic 1 DMO similarly noted:

So [the champion] really likes the program and [they] really like the program to be performing well … like I said that [champion] has got a good working relationship with [their] fellow workers, both the nursing and the clinicians and even the data clerk. So the relationship is quite well and they are all supportive because of that as well; and knowing that [the champion’s] success is their success. – Clinic 1, DMO

When Clinic 1 staff described implementation barriers, their accounts seemed to minimize the barriers and focus mostly on the clinic’s ability to overcome them. The same champion described the issue of clinician resistance resulting from the increased workload after introducing the MBC intervention:

At first when we were introducing depression screening and management, you know when you are starting a thing you always have hiccups but eventually clinicians welcomed the initiative. Everything went on very well … [One] of the challenges that we had is issues of workload. We were introducing [additional paperwork] then it was like the time that the clinicians should spend with the client or the patient could be increased. Those were the minor things that we experienced and there was that bit of resistance but eventually clinicians welcomed the idea. The program went on well, it wasn’t a major thing … when the program was enrolled we did not experience much problems concerning the clinicians doing the screening. – Clinic 1, Champion

Clinic 1 was selected for the enhanced strategy and participants described the positive impact of the audit and feedback visits; the DMO noted how:

[The audit and feedback visits have] impacted greatly I would say. Because some of the problems that we are solving were noted by them, so I think the [audit and feedback] has helped us; and mostly because they are well scheduled, so to make sure that the next time they come we would have addressed some of the challenges that we’ve had and they would even propose for the solution … So whenever we see there’s a feasible solution, we would adopt that and do that. – Clinic 1, DMO

The DMO again reflected their engaged leadership referring collectively to themselves and the NCD clinic when describing the positive impact of the audit and feedback visits (i.e., “the problems that we are solving”). The engagement of the DMO seemed to facilitate an environment where the champion strategy and the MBC intervention were supported and rewarded, ultimately leading to the site’s ability to overcome contextual barriers that might otherwise threaten implementation. Overcoming barriers helped the champion + audit and feedback strategy to function as intended, facilitating adequate implementation of the MBC intervention.

However, this level of engagement was not observed among hospital leaders at all sites. Less engaged leaders exhibited a poorer awareness of the champion’s duties, were less present at the NCD clinic, tended to prioritize diabetes and hypertension care above depression care, were less aware of implementation barriers, and less active in facilitating solutions to overcome them. The absence of strong leadership fostered environments where champions acted alone in their efforts to overcome implementation challenges, often taking on the burden of the depression screening and treatment themselves. The series of quotes below illustrate implementation at Clinic 5, which scored the third lowest on total champion fidelity (77%, shared with Clinic 9) and second lowest on total MBC fidelity among all 10 clinics (61%) (Table 2). The quote below, from the Clinic 5 champion, a site also assigned to the enhanced strategy, describes the lack of leadership engagement:

Sometimes, when you speak yourself, [the DHO or DMO] might think that you are just saying a lot of stories there. So, if they hear from someone who is not working there [like the audit team], at least maybe it makes sense … I do not have any control. I just leave it the way [the auditors] have said it. – Clinic 5, Champion

The Clinic 5 champion’s description focuses first on their perception of the DMO or DHO as devaluing their account of implementation challenges, dismissing them as “stories.” Because the audit and feedback team are external supervisors coming from outside the clinic, the champion feels the DMO and DHO perceive their account of implementation challenges to be more objective or trustworthy compared to their own. The lack of trust between the hospital leaders and champion at this site contrasts with the open communication and shared problemsolving described by the Clinic 1 champion and DMO. Clinic staff at sites with lower leadership engagement tended to also describe worse implementation climates. The same Clinic 5 champion also noted how the champion strategy and MBC integration felt unsupported at their clinic:

Clinic 5 champion: I still feel that we should have enough staff to work there. Because the [depression] screening is just hectic, you cannot just rush and especially whenever there is a patient who has [screened positive for depression], then you take a lot of time.

Interviewer: So who do you think should make sure that there is more staff to work at the NCD clinic?

Clinic 5 champion: The DMO.

Interviewer: Have you ever talked to [them] about this issue?

Clinic 5 champion: Yes, but the problem is the same issue, that there are few people, so there is nothing I can do about that one. And also as I have already said, there are competing priorities among the clinicians, so it is difficult to control those ones.

A provider at Clinic 5 further explained how MBC coverage was impacted by barriers like high workload, lack of provider training, and provider resistance to MBC:

On the negative side, I can say that because [of] the large number of patients that we see on each day of clinic, we find it tough to assist them accordingly because it takes some time to assist one patient because of these [depression] questionnaires … Another problem is that most people whom we expect to help us delivering services to our patients, they are nowhere to be seen, or they run away from this duty. – Clinic 5, NCD Provider

The Clinic 5 champion similarly described the resistance among their fellow NCD providers and further explained how they took on most of the depression screening burden as a result:

Clinic 5 champion: On paper, I think we were supposed to be four or three, but sometimes you will find that I am alone with the nurse and the hospital servants, who assist me doing other things.

Interviewer: So who does the [depression] screening?

Clinic 5 champion: I do the screening myself!

In addition to the champion shouldering the burden of MBC integration themselves, the impact of the audit and feedback visits seemed diminished at sites like Clinic 5 with less leadership engagement and weaker implementation climates. The research staff embedded at Clinic 5 described the fleeting impact the auditors had on implementation:

I see that the week that [the audit team] comes, things change maybe for a month … then things go back as before. If these visitors came more frequently, things would change because people would be afraid, and conduct procedures correctly. Things change positively for that month, and even the patients assessed for depression for that month increases and we see that there is change; but once they forget [about the audit visit] things go back to when things were not going on well. – Clinic 5, Embedded research staff member

The research staffer’s account of the positive but temporary impact made by the auditors seemed to track with challenges regarding the champion strategy at Clinic 5. The research staff’s account of provider depression screening being motivated by fear of the audit team contrasts heavily with the Clinic 1 DMO’s account of working collectively alongside the auditors to improve MBC integration.

Clinics with lower leadership engagement, combined with climates where implementation was less supported and rewarded, seemed to ultimately block those sites from overcoming the same implementation barriers that were successfully addressed at other sites, saddling the champion with the task of MBC integration themselves and trending towards poorer champion strategy and MBC fidelity compared to sites with higher leadership engagement and implementation climates.

3.3. Shared MBC screening approach

Our qualitative analysis identified the themes described above, where champions who described environments with lower leadership engagement and implementation climate also described shouldering the burden of MBC implementation themselves. This finding was not apparent when restricting our attention to our initial quantitative measures of the proportion of clinicians on duty trained in MBC (clinic coverage), and overall clinic-level depression screening. Champions included in the qualitative sub-sample who described taking on the burden of depression screening themselves, suggesting ineffective implementation of the champion strategy, came from three sites that each scored at or above the study average for clinic coverage fidelity (Clinics 5, 8, and 9). To understand how champions at sites with seemingly adequate proportions of MBC trained clinicians on duty could report taking on the burden of screening themselves, we examined the extent to which screening was shared equally across clinicians at each site.

Using Clinic 5 as an example, 98% of all providers were trained in MBC (training coverage), and the champion scheduled MBC trained providers to staff the clinic on 89% of clinic days during the study period (clinic coverage) (Table 2). Table 4 displays the number and proportion of patients screened by Clinic 5’s 20 MBC trained providers. The table reveals that only two providers (C5-030 and C5-002, the site’s champions at different points during the study) accounted for 73% of all MBC screenings during the study period.

Table 4.

Clinic 5 depression screening by provider.

Provider	Provider Screenings	Site Screenings	Screening %
C5-030	1105	1992	55%
C5-002	355	1992	18%
C5-022	261	1992	13%
C5-018	86	1992	4%
C5-023	45	1992	2%
C5-013	33	1992	2%
C5-008	31	1992	2%
C5-028	30	1992	2%
C5-020	23	1992	1%
C5-009	12	1992	1%
C5-021	11	1992	1%
C5-003	0	1992	0%
C5-011	0	1992	0%
C5-015	0	1992	0%
C5-019	0	1992	0%
C5-024	0	1992	0%
C5-025	0	1992	0%
C5-026	0	1992	0%
C5-031	0	1992	0%
C5-038	0	1992	0%

We converted the data in Table 4 to the shared screening coefficient, a metric analogous to the Gini coefficient classically used to summarize the equality or inequality of wealth distribution in a society. This metric summarizes the extent to which screening was shared equally or unequally across all trained providers. A coefficient of 1 indicates perfectly shared screening with all providers screening an equal number of patients, whereas a coefficient of 0 indicates perfectly unequal screening with one provider screening all patients.

Table 5 displays shared screening coefficients by site, ranging from 0.14 to 0.55 ($\overline{\mathrm{x}}=0.29$, sd = 0.12). Clinic 5 ranked among the 3 clinics with the least equal screening among clinic providers. Interviews from staff at Clinic 5 above described how factors like poor leadership engagement and implementation climate contributed to the site’s champions taking on the bulk of MBC screening themselves. Interviewees at the other two clinics with the least equal screening from the qualitative subsample (Clinics 8 and 9) described the same phenomenon.

Table 5.

Shared screening coefficient by site.

Clinic	Shared screening coeff.
7	0.14
10	0.16
5	0.16
2	0.27
8	0.28
9	0.29
3	0.32
1	0.34
4	0.44
6	0.55
$\overline{\mathrm{x}}$	0.29
S	0.12

Clinics 6, 4, and 1 represented the most equally distributed depression screening among their providers. Descriptions of Clinic 1’s high level of leadership engagement and implementation climate, and its impact on MBC implementation were described above, while we describe Clinic 4 as an outlier later in this section. Clinic 6 represents a special case as their champion and the majority of their NCD providers were not MoH staff like other study clinics, but rather staff supplied by a local health service and research NGO. Because of their non-MoH status, Clinic 6 staff were more accustomed to clinical innovations, and better able to enact solutions to implementation barriers without needing to consult hospital leadership. It is likely that the combination of a strong implementation climate and a leadership structure localized within the NCD clinic (rather than at the hospital level), contributed to their ability to distribute depression screening more equally among providers compared to other clinics in our study.

While not an a priori hypothesis, we examined the correlation between overall champion fidelity and the shared screening coefficient, postulating the two to be positively correlated. The analysis resulted in moderate positive correlation, but the findings were not significant (r = .58, 95% CI[−0.06, 0.89]) (Table 3). We also hypothesized that a better shared screening coefficient would share a positive correlation with MBC coverage, or the proportion of patients screened for depression. This analysis returned another moderate positive correlation, but the findings were also not significant (r = 0.41, 95% CI [−0.16,0.55]) (Table 3). Figs. 1 and 2 depict the relationship between overall champion fidelity and the shared screening coefficient, as well as the shared screening coefficient and MBC coverage respectively through bi-variate scatterplots.

Fig. 1.

Champion Fidelity, Shared Screening Coefficient (as %) Bi-variate Scatter Plot

(Orange = basic, blue = enhanced).. (For interpretation of the references to colour in this figure legend, the reader is referred to the Web version of this article.)

Fig. 2.

Shared Screening Coefficient (as %), MBC Coverage Bi-variate Scatter Plot (Orange = basic, blue = enhanced). (For interpretation of the references to colour in this figure legend, the reader is referred to the Web version of this article.)

Visual inspection of Fig. 1 displays the positive relationship between the two variables, as well as how much of a challenge shared screening posed for study sites. Fig. 2 similarly shows a positive relationship, this time between the shared screening coefficient and MBC coverage. The figure effectively summarizes our main study findings. Closer inspection of Fig. 2 yielded 3 clusters of sites and 1 outlier in Clinic 4.

While other sites included in the qualitative sample seemed to dichotomize more neatly into higher or lower leadership engagement, implementation climate, champion fidelity, and MBC fidelity, Clinic 4 required a more nuanced interpretation. Review of clinical records indicated Clinic 4 to have the highest workload among all study sites, averaging 856 patients/month during the study period, 2.45 standard deviations above the study average (Table 2). During their interview, the Clinic 4 DMO exhibited qualities of a more highly engaged leader and attempted to facilitate MBC through motivational talks with NCD providers. However, our quantitative results found Clinic 4 to have the second lowest depression screening (MBC coverage) (26%, $\overline{\mathrm{x}}=63\%$, sd = 26%) and only slightly above average clinic coverage (83%, $\overline{\mathrm{x}}=81\%$, sd = 12%) among all sites (Table 2). When interviewed, Clinic 4’s NCD providers described a desire for structural solutions to reduce workload like (WHO, 2014) operating the clinic daily, rather than just twice per week and (Moussavi et al., 2007) assigning providers to work at the clinic for a full month to stabilize the workflow, rather than defaulting to day-to-day assignments. While the Clinic 4 DMO was knowledgeable about the champion’s duties and attempted to overcome implementation barriers through motivational talks with the NCD providers, our results suggest a site with a supportive leader who struggled to activate effective solutions to overcome an exceedingly high provider workload.

The first cluster comprised Clinics 1 and 6, who represented the 1st and 2nd highest MBC coverage scores, and 3rd and 1st highest shared screening coefficients respectively, among all sites. Qualitative interviews at these sites described strong leadership and implementation climates and their ability to overcome implementation barriers. Clinics 2, 3, 8, 9, and 10 constituted the second cluster which operated towards the lower-middle bounds of MBC coverage and the shared screening coefficient. Clinics 8 and 9 were part of the qualitative sample. Participants at these sites aligned in their descriptions of lower leadership engagement and implementation climate where the champion strategy and MBC were less supported, rewarded, and expected. Clinics 5 and 7 make up the third cluster representing the lower bounds of both indicators. Qualitative results from Clinic 5 were described above and similarly characterized lower leadership engagement and implementation climate compared to sites in the highest cluster. Clinic 7 was unfortunately not included in the qualitative sample, but study staff reports and study records detailed instances of champion turnover and provider resistance that may have driven down MBC coverage.

The lack of leadership engagement and implementation climate precluded sites in the second and third clusters from overcoming implementation challenges, harming champion strategy fidelity, and resulting in champions taking on the responsibility of depression screening and treatment themselves. While all sites described similar barriers to MBC implementation, like increased workload and provider resistance, sites in the first cluster who described strong leadership engagement and implementation climates activated solutions to overcome those barriers. In doing so, champions at Clinics 1 and 6 described the ability to operate more effectively in their roles, and quantitative results support more equally shared depression screening among their providers, and the highest MBC fidelity scores among all clinics.

4. Discussion

Our mixed methods results suggest that strong leadership engagement and implementation climate tended to positively impact champion fidelity as well as MBC fidelity by facilitating solutions to barriers otherwise blocking implementation. Results also describe how better champion strategy fidelity may ultimately have led to better MBC fidelity, suggesting a possible moderation relationship.

In prior literature, intervention fidelity research has established a moderation effect for intervention fidelity on the relationship between an intervention and its intended health impacts, such that interventions carried out with higher fidelity tend to yield better clinical outcomes compared to the same interventions delivered with lower fidelity (Dane and Schneider, 1998; Dusenbury et al., 2003b). Our results allude to the same type of moderation relationship with ISF. Future research, especially studies powered to detect a moderation effect of ISF on the relationship between the strategy and its intended outcomes, can help to further illuminate this relationship.

Despite the critical role ISF might play along causal pathways, a 2015 scoping review (Slaughter et al., 2015) concluded that fidelity domains were on average inadequately measured across implementation trials, and that few reports of ISF existed (Slaughter et al., 2015). The authors also found that the quality of ISF reporting demonstrated a statistically significant decline over time (Slaughter et al., 2015). The lack of fidelity description regarding implementation strategies contributes to an environment where it is not always clear which strategies were performed nor how well they were performed, ultimately challenging researchers’ abilities to assess the likelihood of a Type-III error. Despite numerous difficulties in assessing ISF, our authorship group and others have called for its advancement (Akiba et al., 2021, 2022). The present study may serve as an example of how ISF assessment might pragmatically integrate within a trial of implementation strategies.

Our mixed methods results also described the importance of leadership engagement, its interface with implementation climate, and its ability to facilitate solutions to implementation barriers, ultimately impacting the level of champion strategy fidelity and MBC fidelity. Leadership engagement is defined as leaders’ attitudes, agendas, and active engagements with an intervention, all of which are critical to implementation success (Nielsen; Nielsen and Randall, 2012; Aarons et al., 2018; Harvey et al., 2011; Kyratsis et al., 2012). Implementation climate is the extent to which an intervention is rewarded, supported, and expected within an organization (Weiner et al., 2011). The impact of leadership on facilitating strong implementation climates, favorable provider attitudes regarding intervention adoption, and improved patient outcomes are clear in the health literature (Green et al., 2014; Aarons et al., 2012; Corrigan et al., 2000). Considering this literature and our findings, champion strategies may look to harness the power of organizational leaders to improve champion strategy fidelity and implementation outcomes in this setting. For example, Skar et al. (2021) found that a blended implementation strategy, Leadership and Organizational Change for Implementation (LOCI), significantly increased leadership and implementation climate within mental health clinics in Norway, laying the groundwork for successful implementation of mental health interventions (Skar et al., 2022). Alongside supports from additional strategies, improving our understanding of and response to factors that influence champion fidelity will be critical going forward. Bunce et al. (2020) documented the operationalization of a champion strategy within US-based community health centers to adopt guideline-concordant cardioprotective prescribing among providers. They found the model most successful when champions were (WHO, 2014) engaged, influential, credible, and possessed the capacity for their role, and (Moussavi et al., 2007) experienced organizational support for guideline adoption (Bunce et al., 2020). Our recommendations align with the authors’ ultimate call for implementation researchers to attend to barriers that might threaten champion fidelity at multiple levels by enlisting appropriate supports to overcome them (Bunce et al., 2020). Similarly, Hoekstra et al. (2017) assessed the trajectory of fidelity scores from an evidence-based physical activity program within Dutch rehabilitation centers over a 3-year period. The authors found that fidelity scores varied by site and additionally, over time (Hoekstra et al., 2017). Providers at sites that sustained high intervention fidelity supported, appreciated, and felt a high degree of compatibility with the intervention while effectively attending to implementation barriers (Hoekstra et al., 2017). The variation in our fidelity results share similarities with these findings. While NCD clinics in our study experienced similar barriers like provider resistance and increased workload, sites with higher leadership engagement and implementation climates were able to overcome those barriers and achieve high champion and MBC fidelity, while sites with lower leadership engagement and implementation climates were not. We echo Hoekstra et al. (2017)’s conclusion that successful scale-up requires tailoring responses to barriers at the local organizational level, relative to a setting’s position, size, and circumstances (Hoekstra et al., 2017) Combined with our findings regarding the importance of leadership engagement and implementation climate, pairing champion strategies with additional strategies that focus on supporting local leaders like the aforementioned LOCI or perhaps implementation facilitation may help achieve and sustain high champion fidelity.

We created the shared screening coefficient to reflect the extent to which depression screening (MBC coverage) was shared among clinicians at each site. Clinics’ shared screening coefficients averaged just .29 depicting the challenge of equal screening. Some research exists regarding the harmful impact of unequal caseloads on implementation. Strauss et al. (2009) integrated a screening and treatment intervention for alcohol use disorder into routine HIV care and found that providers with smaller caseloads were better able to implement the intervention compared to providers with larger caseloads (Strauss et al., 2018). Shapiro et al. (2012) similarly found that when an evidence-based parenting intervention to prevent child maltreatment was not properly integrated within providers’ caseloads, it decreased their use of the intervention (Shapiro et al., 2012). The same authors concluded that fully understanding the time required to implement an intervention, alongside assessing providers’ available time, represent crucial actions that should precede intervention adoption (Shapiro et al., 2012). Future research may look to similarly assess barriers to caseload equality given its potential impact on successful implementation.

This study was limited by a lack of statistical significance testing of our quantitative associations due to the clinic-level nature of the analysis. Conceptualizing fidelity to the basic and enhanced strategies at the clinic level locked our sample size at n = 10. This precluded our ability to carry out psychometric evaluation of our fidelity indicators or hypothesis testing regarding associations between ISF and MBC fidelity. Sample size often challenges fidelity assessment given that trials are rarely powered to detect an effect at the unit of analysis for which fidelity is commonly assessed (Bond and Drake, 2019). Sample size will likely challenge ISF assessment going forward given implementation research’s tendency to treat higher order units like clinics or hospitals as units of analysis (Newhouse et al., 2013). Our use of mixed methods facilitated the assessment of ISF and MBC fidelity through techniques like data merging and triangulation, ultimately enabling a holistic fidelity assessment (Davidov et al., 2020; Azano et al., 2011; Lorencatto et al., 2016; Williams et al., 2020; Nelson et al., 2014). Future research to further develop ISF might include a priori power analyses to detect an effect to facilitate more robust quantitative analyses, or further refine mixed methods approaches.

This study also exhibited several strengths. First, our mixed-methods assessment of ISF adds to a nascent but critical area of inquiry within implementation research by not only assessing ISF but further assessing its relationship with intervention fidelity (Akiba et al., 2021). Our multiple mixed methods strategies, including simultaneous and iterative/sequential approaches to triangulate findings, facilitated the enhancement of qualitative and quantitative analyses at different stages. Second, we took a theory-informed approach, starting with the expansion of the ‘dose’ category of Proctor et al. (2013)’s specification recommendations to include additional fidelity components informed by Carroll et al. (2007)’s conceptual framework for implementation fidelity (Proctor et al., 2013; Carroll et al., 2007). These actions combine calls within the implementation literature to improve both the specification and fidelity assessment of implementation strategies (Lewis et al., 2020; Akiba et al., 2021; Boyd et al., 2018; Bunger et al., 2017; Haley et al., 2021; Miller et al., 2021). Challenges to this approach included time intensive and costly quantitative and qualitative data collection, and a lack of uniform data collection via observation (often considered the gold standard for fidelity assessment). Such factors regularly challenge fidelity assessment and are likely to act as barriers for similar efforts in the future (Akiba et al., 2022; Schoenwald et al., 2011; Schoenwald, 2011). However, several factors ultimately facilitated our fidelity assessment including the availability of multiple process data indicators, and a mix of data sources (e.g., observation by trained research staff, provider self-report, qualitative interviews). Our combination of fidelity-focused implementation strategy specification and utilization of mixed methods may serve as a reference point for future ISF assessments to improve and refine.

5. Conclusions

This study utilized a novel, theory-informed, mixed methods approach to assess fidelity of a champion and champion + audit and feedback implementation strategy meant to improve fidelity of the MBC depression intervention across 10 NCD clinics in Malawi. We found that ISF might moderate the relationship between the strategies and MBC fidelity. We also determined that fidelity to the strategies and to the MBC intervention varied by site due to the contextual influences of leadership engagement and implementation climate. These results may serve as a building block for future efforts looking to assess ISF and evaluate its relationship with other key variables. Doing so may ultimately improve the quality of implementation research and the replicability of implementation strategies in practice settings.

Abbreviations

AMI

Amitriptyline

DHMT

District Health Management Team

DHO

District Health Officer

DMO

District Medical Officer

EBI

Evidence-based Intervention

FLU

Fluoxetine

ISF

Implementation strategy fidelity

LMICs

Low and middle-income countries

MoH

Ministry of Health (Malawi)

NCDs

Non-communicable diseases

PHQ-9

Patient Health Questionnaire 9

RTA

Reflexive Thematic Analysis

SHARP

Sub-Saharan Africa Regional Partnership for Mental Health Capacity Building Trial

SRA

Suicide risk assessment

YLD

Years lived with disability