Partnership for Implementation of Evidence-Based Mental Health Practices in Rural Federally Qualified Health Centers: Theory and Methods

Abstract

Background

Mental health and substance abuse are among the most commonly reported reasons for visits to Federally Qualified Health Centers (CHCs), yet only 6.5% of encounters are with on-site behavioral health specialists. Rural CHCs are significantly less likely to have on-site behavioral specialists than urban CHCs. Due to this lack of mental health specialists in rural areas, the most promising approach to improving mental health outcomes is to help rural primary care providers deliver evidence based practices (EBPs). Despite the scope of these problems, no research has developed an effective implementation strategy for facilitating the adoption of mental health EBPs for rural CHCs.

Objectives

To describe the conceptual components of an Implementation Partnership that focuses on the adaption and adoption of mental health EBPs by rural CHCs in Arkansas.

Methods

We present a conceptual model that integrates seven separate frameworks: 1) Jones and Wells’ Evidence-Based Community Partnership Model, 2) Kitson’s Promoting Action on Research Implementation in Health Services (PARiHS) implementation framework, 3) Sackett’s definition of evidence-based medicine, 4) Glisson’s organizational social context model, 5) Rubenstein’s Evidence-Based Quality Improvement (EBQI) facilitation process, 6) Glasgow’s RE-AIM evaluation approach, and 7) Naylor’s concept of shared decision making.

Conclusions

By integrating these frameworks into a meaningful conceptual model, we hope to develop a successful Implementation Partnership between an academic health center and small rural CHCs to improve mental health outcomes. Findings from this Implementation Partnership should have relevance to hundreds of clinics and millions of patients, and could help promote the sustained adoption of EBPs across rural America.

Introduction

Federally Qualified Health Centers (CHCs) are the nation’s largest and fasting growing primary care (PC) network, providing services to over 20 million Americans living in medically underserved inner-city neighborhoods (47%) and rural communities (53%). CHCs provide services to 10% of rural Americans, 14% of minorities, and 20% of the uninsured. CHCs represent the country’s PC safety net—they treat all patients equally, serve all patients regardless of ability to pay, and aim to eliminate health disparities. Mental health problems are common among CHC patients. In our OUTREACH study (R01 MH076908), the prevalence of depression was noted to be 16% in rural CHC settings, and mental health and substance abuse are among the most commonly reported reasons for visits to CHCs, yet only 6.5% of encounters are with on-site behavioral health specialists.¹ Rural CHCs are significantly less likely to have on-site behavioral specialists than urban CHCs,² and linkages with Community Mental Health Centers are often inadequate.³ Despite the scope of these problems, no research has developed an effective implementation strategy for facilitating the adoption of mental health evidence based practices (EBPs) for rural CHCs.

Although mental health outcomes are equally poor in rural and urban areas, quality improvement (QI) strategies should not necessarily be the same.⁴ Due to the lack of mental health specialists in rural areas, one promising approach to improving mental health outcomes is to help rural PC providers deliver EBPs.^2,4-6 Compared to urban areas, PC providers in rural areas play an even larger role in the de facto mental health care system,^6-8 yet rural PC providers frequently lack the expertise, time, and resources to effectively treat mental health and substance use disorders. The relatively large patient panels of rural PC physicians ⁹ and short encounter times¹⁰ make it challenging to provide high quality mental health care during an encounter with multiple competing demands.¹¹ Moreover, the linkages between rural PC practices and distant specialty mental health care practices are weak in most rural areas, making referrals infeasible,¹⁰ and use of off-site mental health specialists unlikely.¹²

Another major barrier to implementing EBPs in small rural clinics is that efficacy and effectiveness researchers typically design and test EBPs in large urban clinics. Thus, the efficacy and effectiveness trials that constitute the evidence base are not necessarily generalizable to rural clinics serving disadvantaged populations, and the interventions are not necessarily feasible to implement. Organizational theory and experience suggests that adaptation to local context is critical to adoption and sustainability.^13-16 Because researchers cannot conduct randomized controlled trials for every intervention in every possible clinical setting, it is necessary for CHCs to develop an internal capacity to adapt EBPs based on their own preferences, needs and capacities and to evaluate the clinical impact of adoption using locally collected data. Moreover, the availability of timely, locally collected outcomes data should promote EBP sustainability, perhaps even more than external data from published randomized controlled trials.^17,18 However, small rural clinics often lack the centralized infrastructure (e.g., dedicated staff with QI or evaluation expertise) needed to coordinate implementation evaluation efforts.^19,20

In this article, we describe the conceptual components of an Implementation Partnership that focuses on the adaptation and adoption of mental health EBPs by rural CHCs in Arkansas. While the overarching conceptual components of this Implementation Partnership would be the same in urban CHCs, we focus on rural CHCs because of the higher level of adaptation required in those settings. With funding from the National Institutes of Mental Health (R24 MH085104), the Implementation Partnership embeds research faculty from an Academic Health Center (the University of Arkansas for Medical Sciences [UAMS] Department of Psychiatry) into the Arkansas community health centers and the Arkansas primary care association, the Community Health Centers of Arkansas, Inc. (CHCA), which is responsible for coordinating quality improvement activities in CHCs. Scholars and policy makers have argued that funding partnerships to embed researchers within clinical organizations is a promising approach for implementing EBPs and advancing the emergent field of implementation science.^21-29

Methods

The conceptual model for this rural Implementation Partnership is presented in Figure 1. Partnership building is guided by the principles of the Evidence-Based Community Partnership Model.²² Implementation strategies are based on the Promoting Action on Research Implementation in Health Services (PARiHS) model,³⁰ including facilitation via the Evidence Based Quality Improvement (EBQI) model.^31,32 Implementation outcomes are measured using the RE-AIM framework.^33-36 Facilitation processes and implementation outcomes both reciprocally influence the success of the partnership. The evaluation of the Implementation Partnership itself is guided by the principles of shared decision-making.³⁷ This project received approval from the University of Arkansas for Medical Sciences Institutional Review Board.

Figure 1. Conceptual Frame for an Implementation Partnership between Community Health Centers and an Academic Health Center.

Partnership Building

One of the main barriers to EBP implementation has been the lack of collaborative efforts between academic researchers and community stakeholders, and the subsequent failure of researchers to address community needs.²³ Research conceived and conducted within a clinical-research partnership has the potential to increase the rate of uptake of research findings dramatically.³⁸ The President’s New Freedom Commission on Mental Health specifically charges researchers to work in conjunction with community partners to bridge the gap between science and service.³⁹ Participatory research within a partnership builds on the strengths of the various partners (researchers, clinicians, patients, administrators). The reciprocity, equity and transparency of participatory research over the long run is needed to build the trust required to understand and address health disparities among disadvantaged populations.⁴⁰ Despite strong calls for participatory research, there is a paucity of systematic studies on collaborative partnerships between mental health services researchers and rural community stakeholders.^37,41 Although embedding researchers within clinical organizations is a highly promising approach to understanding and facilitating implementation, we know little about how to make such partnerships effective.²⁵

Differences in goals, methods, and timelines between scientists and community stakeholders create fundamental challenges to sustaining partnerships. Scientists are expected to carefully apply meticulous research methodologies to generate incremental advances in knowledge while providers are expected to quickly solve acute problems in their clinics.⁴² Partnerships must be forged that align the goals, methods and timelines of researchers with those of community stakeholders. Successful collaborations require researchers to revise expectations concerning scientific rigor (i.e., internal validity).^17,18 To be successful, the researchers in the Implementation Partnership will need to demonstrate flexibility in terms of specifying goals, methods and timeframes of the research to fit the needs and priorities of the community stakeholders.^43,44

The conceptualization of the Implementation Partnership is based on Wells’ Evidence-Based Community Partnership Model which “…differs from a fully participatory process, which could lead to more sustained change, but not necessarily to use of evidence-informed strategies.” This integrated model blends the traditions of evidence-based QI and participatory research.²² The Implementation Partnership is structured according to the 12 guiding principles outlined by Jones and Wells: 1) shared decision-making and power between academics and community stakeholders; 2) use of written agreements and standard operating procedures; 3) frequent communication and direct forms of conflict resolution; 4) transparency; 5) understandability; 6) resource sharing; 7) respect for community values and timeframes; 8) scientific integrity; 9) academic productivity; 10) mutual reliance; 11) awareness of history and culture; and 12) development of capacity and leadership among community stakeholders.⁴³

The structure of the Implementation Partnership includes a Steering Committee, EBQI Teams (focused on specific clinical disorders), two Advisory Boards and six Support Cores: 1) Clinical Training Core, 2) Data & Technology Core, 3) Evaluation Core, 4) Technical Writing Core, 5) Financing Core, and 6) Sustainability Core. The National Advisory Board includes clinical research experts in relevant clinical disorders and in implementation science. The Community Advisory Board includes members of the patient-majority community boards, including representation from minority groups. Our academic-community partnership initially developed through the NIMH-funded OUTREACH study (R01 MH076908). We developed this study to determine whether it is more effective for small rural PC clinics to provide collaborative care services on-site or to contract with an off-site depression care team via telemedicine. To initiate the partnership formation, we first presented the proposed study to the CHCA Executive Director and Clinical QI Manager and later presented a preliminary study design to the CHC Medical Directors at a quarterly Provider Board meeting. In partnership with local CHC staff, the OUTREACH team adapted the collaborative care intervention for CHCs and jointly developed the study design. This participatory process resulted in a more feasible study design. Through this RCT, the on-site research and clinical staff in the CHCs became more engaged with the academic medical researchers leading to our strong current academic-community partnership.

A Steering Committee comprised of faculty from the academic health center, staff from the Arkansas primary care association (CHCA), and Arkansas community health center staff/patients governs the Implementation Partnership. A key component of successful participatory research models is written agreements that outline the partnership’s mission, common values, responsibilities, leadership structure, operating procedures, data ownership, product review, guidelines for publications and press releases, and conflict resolution procedures. We base our Implementation Partnership’s operating procedures on the Collaboration Agreement developed by Jones and Wells.⁴³ The Steering Committee will conduct strategic planning, based on ongoing formative evaluation of the partnership to revaluate priorities and progress towards goals, conduct annual review of budget and personnel allocation to ensure they are aligned with priorities, serve as the Data and Safety Monitoring Board, and disseminate findings to stakeholders. It will be critical to disseminate results to CHC administrators, providers, patients in a timely manner and to tailor the information to meet the administrative and clinical needs of all the CHC stakeholders.

PARiHS Implementation Framework

In this study, we use the PARiHS framework to guide our implementation strategy.³⁰ The PARiHS framework proposes that successful implementation of EBPs is a function of: 1) evidence, 2) context, and 3) facilitation.⁴⁵ Researchers have traditionally considered evidence to be the clinical outcomes of interventions tested in randomized controlled efficacy trials and then re-evaluated in effectiveness studies conducted under “real world” conditions.²¹ We adopt Sackett’s definition of evidence-based medicine: “the conscientious, explicit and judicious use of current best evidence in making decisions about the care of the individual patient. It means integrating individual clinical expertise with the best available external clinical evidence from systematic research.”⁴⁶ We adopt this definition for our proposed participatory research because it weighs equally the importance of research findings and clinical experience. We also argue that clinicians and policy makers will be more likely to rely on locally generated “practice-based evidence” than on the scientifically generated evidence base. ^21,22,47-50

Organizational Context, including organizational culture, climate, and capacity,^51-58 is expected to influence the adoption of EBPs. Culture is generally conceptualized as the values and expectations of an organization.⁵⁹ Climate is generally conceptualized as activities and experiences of workers.⁵⁹ Capacity is generally conceptualized as the ability to make changes (e.g., resources, skills, etc.).⁶⁰ We expect that, compared to large urban clinics, small rural clinics have very different organizational culture, climate and capacity. CHC culture and capacity is being assessed quantitatively using the Organizational Social Context survey^59,61 which is being administered anonymously to clinical staff. During site-visits, researchers are also conducting non-participant observations of clinical and administrative operations to better understand each program’s common and accepted ways of doing things and to get a sense of staff cohesion, conflict, and burnout. In additional, key informant interviews with clinical and administrative leaders are addressing organizational capacity for QI and organizational culture and climate. The quantitative and qualitative data is being summarized into organizational profiles that will be used to assess capacity for EBP adoption, to customize facilitation methods for each CHC, and to aid in interpretation of implementation outcomes.

Facilitation typically involves a set of integrated strategies including identifying and engaging key stakeholders at all organizational levels, problem identification and resolution, assistance with technical issues, academic detailing, marketing, staff training, audit and feedback, quality improvement, and fostering role modeling.^62,63 We are using a well validated facilitation method known as Evidence-Based Quality Improvement (EBQI).^64,65 EBQI was developed by Rubenstein and colleagues based on the findings of the Mental Health Awareness Project, which compared two quality improvement strategies for depression in PC.⁶⁶ Clinics were randomized to a top-down or a bottom-up model.⁶⁶ The top-down approach involved centralized experts implementing depression evidence-based practices with some input from local PC staff. The bottom-up approach involved local clinical staff implementing depression evidence-based practices with some input from experts. The bottom-up quality improvement teams had both the best and worst outcomes in terms of fidelity to the evidence base.⁶⁶ This finding suggests that the bottom-up approach has the best potential for quality improvement, but is subject to substantial variation depending on local climate, culture and capacity.⁶⁶ Based on these findings, the EBQI model was developed which involves both centralized strategic decision-making and local tactical decision-making.⁶⁷ There is a growing consensus among implementation experts^68-70 and frontline clinicians and managers ^66,67,71 that quality improvement strategies that incorporate both top-down and bottom-up approaches hold the most promise for sustained implementation of evidence based practices.

In EBQI, both researchers (clinical experts, implementation experts) and local staff participate fully in the quality improvement process, with the researchers facilitating rather than dictating implementation efforts.^66,67,71 Thus, EBQI is intended to foster a researcher/clinician partnership that promotes buy-in.^72,73 While emphasizing the involvement of outside experts and empirical evidence, EBQI stresses that an organization’s own healthcare professionals and staff are best positioned to improve their systems.⁷² Clinicians and administrators contribute local knowledge needed to tailor the evidence based practice for their own particular needs and organizational capabilities. Researchers contribute knowledge of the evidence base and tools needed for successful implementation. In addition to providing expertise, researchers in the EBQI model also facilitate problem solving and provide ongoing technical support. EBQI also emphasizes continuously revising the adapted evidence based practice based on feedback during Plan-Do-Study-Act cycles, and thus should lead to adapted evidence based practices that are robust, user-friendly, and feasible to deploy in real-world practice settings.

Evaluation of Implementation Outcomes

The evaluation of implementation outcomes is based on the RE-AIM Framework (Reach, Effectiveness, Adoption, Implementation, Maintenance).^33-36 Adoption represents the absolute number/proportion of staff who use the EBP.³³ Reach represents the absolute number/proportion of eligible patients who receive the EBP.³³ Implementation represents the fidelity of the EBP as implemented in routine care.³³ Effectiveness represents the clinical impact (on patient outcomes) of the EBP as implemented in routine care settings.³⁵ Maintenance represents the degree to which the implementation of the EBP is sustained.³³ To have an impact on health of the target population, an EBP must be adopted by providers, reach a large proportion of the targeted patient population, be implemented with fidelity, effectively improve outcomes, and be maintained after research funds are withdrawn.

For each adopted EBP, implementation outcomes initially will be measured during a pilot test conducted at one clinic of each CHC system. Once the Plan-Do-Study-Act cycle is completed, an implementation trial will be conducted in which clinics will being randomized to implementation sites or wait list control sites. CHC staff will collect all implementation outcome data during the pilot tests and the implementation trials. The Data and Technology Core will develop procedures for collecting, tracking, storing and analyzing data for these evaluations, as well as evaluating and deploying technologies to support data collection and analysis efforts. These implementation outcome data will provide the CHCs with internally valid local evidence about the adapted EBP in a timely manner. In Table 1, we display how both the implementation framework and evaluation will be used in an example demonstrating the implementation of an EBP for alcohol use disorders in rural CHCs.

Table 1.

PARiHS Implementation and RE-AIM Evaluation Frameworks: An Example with Alcohol Use Disorders (AUDs)

Implementation Framework (PARiHS)	Examples: Implementation of an EBP for Alcohol Use Disorders (AUDs)
Evidence	Extensive literature review of RCTs of brief alcohol interventions in diverse primary care settings and patient populations. Inclusion of expert consultants knowledgeable in brief interventions for AUDs.
Context	Researchers will conduct site visits involving key informant interviews and surveys to understand the context of each individual clinic. Interviews with providers will collect information about current clinic procedures for detecting and managing AUDs, how well these clinic procedures are working and how to improve these procedures. Interviews with patients with identified AUDs will elicit their experiences and preferences concerning receiving assistance with their drinking.
Facilitation using Evidence-Based Quality Improvement (EBQI)	Plan – Based on the needs assessment, UAMS researchers will develop an implementation template that will sequentially outline the decisions and tasks needed to adapt and implement the EBPs. The EBQI Team will need to integrate the brief alcohol intervention protocols into existing care manager activities. Do – Each individual CHC will select one provider in one clinic to Pilot the adapted EBP, for a total of six pilot PC providers. The adapted EBP will be piloted for at least 6 months. Different EBPs may be piloted at each site. Spread of innovations developed at each clinic could naturally happen in monthly statewide meetings of providers involved in the implementation partnership. Study - UAMS researchers will conduct a formative evaluation during the Pilot and report findings to the EBQI Team. Grand tour questions for providers will include: “What is working and not working with the screening program?” “What is working and not working with the brief intervention program?” Grand tour questions for patients will include: “What are your overall impressions of the screening and brief alcohol counseling program?” Which aspects of the screening and brief alcohol counseling program were helpful? Which were unhelpful? And why?” One month after a positive screen, quantitative data will be collected about whether: 1) they were advised by their PC provider to reduce drinking; 2) contacted by a nurse to provide on-going monitoring and counseling; and 3) referred to an alcohol treatment program. CHCA evaluation staff will compile and report these patient-level outcomes to the EBQI Team. Act - During bi-monthly EBQI meetings, the UAMS researchers will provide qualitative feedback to the team from the semi-structured interviews and quantitative feedback from the patient outcome reports. The EBQI members will discuss these results and will make refinements to the adapted EBP for AUDs. Innovations from other partnership sites might be added if they are appropriate to a site’s context, climate, and culture.
Implementation Outcomes Evaluation Framework (RE-AIM)	Examples: Evaluation of Implementation of an EBP for Alcohol Use Disorders (AUDs)
Provider Adoption	Adoption will be defined as the percent of PC providers who screen their patients and provide advice, brief counseling, outcomes monitoring, and/or referral to those screening positive.
Patient Reach	Reach will be defined by the percent of patients screened for AUDs and the percent of those screening positive who received advice to reduce intake, brief counseling, outcomes monitoring, and/or referral for special alcohol treatment.
Implementation Fidelity	Fidelity will be defined as the accurate scoring of screeners and concordance with brief intervention protocols.
Clinical Effectiveness	Effectiveness will be measured using a site-level randomized study design.
Maintenance	Maintenance as continuous implementation of the EBP during the year long Implementation Trial and intent to sustain implementation after research funding ceases, as measured during brief phone interviews with Executive Directors and Medical Directors.

Evaluation of Partnership

We are conducting a formative evaluation of the Implementation Partnership. Ongoing evaluation will enable us to determine which elements of the partnership are successful and which need improvement. We will use data from the formative evaluation to revise the activities and policies of the partnership in order to make it function more effectively/equitably. The formative evaluation focuses on: 1) the extent of shared decision-making, 2) the efficiency and equity of the EBQI facilitation process, and 3) CHC capacity to conduct their own QI evaluations.

The shared decision-making component of the formative evaluation is based on Naylor’s six elements common to participatory research: 1) identification of clinical issues to target; 2) definition of clinical and research goals; 3) sources of funding; 4) specification of evaluation methodologies; 5) definition of outcome measures; and 6) sustainability of changes and spread to other areas.³⁷ Using the quantitative scale developed by Naylor to measure the extent of researcher/community participation, the Steering Committee members will rate the level of participation for each element along a continuum (researcher driven, cooperation by community partners, equal participation by community partners, or community driven). Researchers will explore these ratings during follow-up key informant interviews. In contrast to formal qualitative analysis methods, we will utilize rapid content analysis techniques so that we can feedback information to the Steering Committee in real time.^74,75

We will also conduct ongoing formative evaluations the EBQI process during the implementation initiatives to insure that they operate efficiently and equitably. We define the EBQI process as being “efficient and equitable” under the following conditions: 1) the correct stakeholder groups are represented, 2) all stakeholders are engaged and participate fully, 3) all stakeholders consider the process balanced across research/clinical needs, 4) participants reach consensus concerning the adaptation of the EBP, 5) community participant’s burden is at an acceptable level, and 6) all stakeholders report high levels of satisfaction with the process.^14,43,44 To explore these issues, we will conduct interviews with the members of the EBQI teams. Again, we will utilize rapid content analysis techniques so that we can feedback timely information to the EBQI Teams and inform their implementation process in real time.^74,75

For the goal of developing an internal capacity for evaluation, the Steering Committee will set yearly goals for CHC staff taking methods courses, receiving training from UAMS faculty, and conducting valid evaluations of the implementation initiatives. We will interview UAMS research faculty conducting the trainings regarding the progress of CHC staff, and will evaluate data analyses conducted by CHC staff for completeness/accuracy.

Conclusions

Previous efforts to implement EBPs for depression in CHCs have proved challenging. For example, the Institute for Health Improvement’s (IHI) Breakthrough Series model used a series of learning sessions and action periods to implement Chronic Care models for specific disorders, including a Depression Collaborative.⁷⁶ The Breakthrough Series allowed CHC staff to consult with national experts and learn from each others’ experiences implementing the Chronic Care models. For the Depression Collaboratives, CHCs collected data on performance measures (e.g., documented PHQ9 reassessment, and 50% reduction in PHQ9 score). The initial outcomes of the IHI Depression Breakthrough Series were good;⁷⁷ however, the CHCs sustained few of the depression care improvements over time.⁷⁸

Considering these less than optimal outcomes, the field clearly needs to develop, test, refine, and disseminate new and more intensive QI models for deploying mental health EBPs in Federally Qualified Health Centers, especially those serving rural populations.⁷⁹ In this paper, we have presented a conceptual model that integrates seven separate frameworks. By integrating these various frameworks into a meaningful conceptual model, we hope to develop a successful Implementation Partnership between an academic health center and small rural CHCs to achieve and sustain improvements in mental health outcomes. Findings and products from the Implementation Partnership in Arkansas should have relevance to hundreds of clinics and millions of patients, and could help promote the sustained adoption of EBPs across rural America.