Different factors identified by stakeholder group for barriers and facilitators to measurement-based care implementation in behavioral health clinics

Abstract

Introduction:

Despite the benefits of measurement-based care (MBC) in the behavioral health setting, there have been difficulties in implementation and low saturation. Although barriers and facilitators to MBC implementation have been identified, research has generally only included the perspective of one stakeholder group. The current study aims to examine the similarities and differences—by stakeholder group—in the identified barriers to and facilitators of implementing MBC in the behavioral health setting.

Method:

A purposeful sampling approach was used to recruit and conduct interviews and focus groups with stakeholders (clinicians, clinic leaders, and administrative staff) from four behavioral health clinics at an academic medical center that is part of a larger healthcare system. The data coding process included a directed content analytic approach whereby the coding team used an iterative process to analyze de-identified transcripts starting with a codebook based on the Consolidated Framework for Implementation Research (CFIR) constructs.

Results:

A total of 31 clinicians, 11 clinic leaders, and 8 administrative staff participated in the interviews and focus groups. There was convergence among all stakeholder regarding which CFIR constructs were identified as barriers and facilitators, but there were differences in the specific thematic factors identified by stakeholders as barriers and facilitators within each of these implementation constructs. The barriers and facilitators that stakeholders identified within each CFIR construct were often connected to their specific role in implementing MBC.

Conclusion:

Collecting information on barriers and facilitators to MBC implementation from the multiple stakeholders involved in the process may enhance successful implementation of MBC given the variation between groups in identified thematic factors. Administrative staff perspectives, which have not been reported in the literature, may be of particular importance in planning for successful MBC implementation.

Introduction

Measurement-based care (MBC) is the systematic collection of data on patient symptoms that are assessed and monitored through patient reported outcome measures (PROMs) to inform clinical care decisions (Resnick et. al, 2020). MBC has been shown to increase the effectiveness of mental health treatments by influencing patient and clinician behaviors (Scott & Lewis, 2015). For example, MBC has been associated with increased patient participation and engagement in the inpatient psychiatric treatment setting (Eisen et al., 2000). In the outpatient setting, MBC has been associated with greater improvement in symptoms, functioning, and quality of life for individuals engaged in psychotherapy (Lambert et al., 2003). Furthermore, data on symptoms collected as part of MBC have been used at a population level to inform policy efforts to increase funding and resource allocation for mental health treatment (Youn et al., 2015). Since MBC provides patient feedback on quality of care it may become a required component of patient care in the United States as the healthcare system moves towards value-based care. (Squitieri et al. 2017)

Despite the multi-level benefits of incorporating MBC into routine clinical practice MBC has not been widely implemented in behavioral health settings (Jensen-Doss et al., 2018). Systematic reviews evaluating MBC implementation have highlighted common barriers and facilitators that may impact widespread use of MBC in behavioral health, but have not included the perspective of multiple stakeholders in the same setting nor to our knowledge the perspective of administrative staff (Boswell et al., 2015; Lewis et al., 2019). A common barrier identified at the clinician-level has been the increased time and effort associated with incorporating measures into care have frequently been cited as reasons for decreased MBC adoption (Boswell et al., 2015; Lewis et al., 2019; Scott & Lewis, 2015), as well as skepticism regarding the utility and clinical applicability of the measures (Lewis et al., 2019). At the level of the behavioral health organization, implementing MBC has been challenged by limited resources to support operational changes and staff training related to MBC, high turnover among personnel, and lack of leadership support (Boswell et al., 2015; Lewis et al., 2019; Scott & Lewis, 2015).

Although knowledge of clinician and organizational barriers to MBC implementation in the behavioral health setting is helpful, there are few examinations of the perspective of multiple stakeholders who work in the same behavioral health setting. Furthermore, the perspective of administrative staff, who often facilitate MBC in larger clinics by giving patients questionnaires when they arrive for their appointment, is an important stakeholder group to include when examining MBC implementation in behavioral health. The absence of input from multiple stakeholders, including administrative staff, may be contributing to the limited implementation of MBC in behavioral health since prior research has shown that stakeholders will have different experiences during the implementation process and thus, different perspectives (Eisman et al., 2021; Svirydzenka et al., 2017). It is therefore important to capture the perspective of all relevant clinical stakeholders who will be part of the implementation process to support more successful MBC implementation in behavioral health (Ferlie & Shortell, 2001; Youn et al., 2019).

The current study aims to expand on the existing literature by assessing for similarities and differences in identified barriers and facilitators to MBC implementation among key stakeholders involved in implementing MBC in the outpatient behavioral health setting, including clinicians, clinic leaders, and administrative staff.

Materials & Methods

The present study is part of a larger observational study that is evaluating the longitudinal effect of behavioral health treatment on mental health symptoms, substance use, and the development/progression of a substance use disorder (SUD). (HEAL Prevention Cooperative, 2021) The study aims to also implement MBC by administering patient reported outcome measures (PROMs) through the electronic health record (EHR) system to assess and monitor mental health symptoms and substance use at initial evaluation and follow-up appointments in behavioral health and SUD clinics within the Department of Psychiatry. The study was approved by the Mass General Brigham Institutional Review Board (IRB).

Setting

The study was conducted at four behavioral health clinics at an academic medical center in an urban location that is part of a larger healthcare system. The clinics included two general behavioral health clinics, one for adults and another for children and adolescents, as well as two specialty clinics for treating SUDs, one for adults and another for adolescents and young adults. These clinics provide a wide variety of services including evaluation, individual and group therapy, and medication treatment. Clinicians include psychiatrists, nurse practitioners, psychologists, licensed independent clinical social workers, and recovery coaches. The clinics use the same EHR system, and infrastructure to support MBC implementation through the EHR system was available within the larger healthcare system (Sisodia et al., 2020).

All of the clinics had some previous experience using PROMs, but none of the clinics had been successful in systematically administering PROMs linked to the EHR at initial evaluation and follow-up appointments (Table 1). Patients (and parents) in all clinics were given PROMs to complete on an electronic tablet or paper by administrative staff when they checked in for their in-person appointment. When PROMs were completed on the electronic tablet, questionnaire responses were automatically synced to the EHR. The child and adolescent behavioral health clinic collected PROMs through parent report of their child’s mental health symptoms at the time of evaluation through an electronic tablet. The adult behavioral health clinic and adult SUD clinic also collected PROMs from patients through an electronic tablet at the time of evaluation. The adult SUD clinic also continued to collect PROMs every 30 days. However, the adolescent and young adult SUD clinic collected PROMs from patients on paper (Table 1). Notably, PROMs collection began at the adolescent and young adult SUD clinic upon its founding in 2009.

Table 1.

Characteristics of the behavioral health clinics involved in the implementation of measurement-based care (MBC) and where individual interview and focus group participants were recruited from.

	Clinic 1	Clinic 2	Clinic 3	Clinic 4
Clinic type	General outpatient psychiatry	General outpatient psychiatry	Substance Use Disorder (SUD) specialty	SUD specialty
Patient age range	Children and adolescents	Adults	Adults	14 to 26 years of age
Clinician types (N)	MD (44) PhD (23) LICSW (1)	MD (117) PhD (51) NP (6) LICSW (5)	MD (7) PhD/PsyD (7) NP (2) LICSW (3) MA (1)	MD (1) NP (1) PhD/PsyD (3) LICSW (1)
Administrators (N)	3	25	2	1
Frequency of clinic-wide meetings	Every three months	Clinic Leadership: Monthly Subgroups within the clinic: weekly to monthly	Weekly	Weekly
Previous MBC launch date	December 2015	July 2016	July 2017	N/A
Previous Patient Reported Outcome Measure (PROMs) collection	Parent report of their child’s mental health symptoms at evaluation only through electronic health record (EHR).	Patient report of mental health symptoms and substance use at evaluation only through EHR.	Patient report of mental health symptoms and substance use at evaluation and follow-up (monthly) through EHR.	Patient report of mental health symptoms and substance use symptoms on paper/pencil.
PROMs collection rates in 2019	1.6%	57.3%	8.4%	N/A
MBC implementation goal	Collect patient report of their mental health symptoms and substance use from children ≥12 years of age at evaluation and follow-up	Collect patient report of mental health symptoms and substance use at evaluation and follow-up	Collect patient report of mental health symptoms, substance use, and risky substance use behaviors at baseline and follow-up	Collect patient report of mental health symptoms, substance use, and risky substance use behaviors at baseline and follow-up through the EHR.

Participants and Sampling Procedure

Between December 2019 and February 2020, a purposeful sampling approach was used to identify and recruit participants for individual interviews and focus groups. Individual interviews were conducted with the clinic leaders of the four behavioral health settings, and focus groups with clinicians and administrators (i.e., administrative staff). Clinic leadership sent an IRB-approved recruitment email to appropriate clinicians and administrators in their clinics. It is also of note that all interviews and focus groups were conducted prior to COVID-19-pandemic related changes in care delivery and practice.

Interview Procedures

A semi-structured interview guide was developed for each stakeholder group of interest: clinicians, leadership, and administrators (see Appendix A for interview guides). The questions were guided by four Consolidated Framework for Implementation Research (Damschroder et al., 2009) domains (Intervention Characteristics, Outer Setting, Inner Setting, and Characteristics of Individuals) to provide constructs of interest in assessing factors that impacted MBC implementation.

To minimize undue bias or influence on the discussions (Onwuegbuzie et al., 2009), each focus group included only participants from each specific participant level (i.e., clinician focus groups only included clinicians), and individual interviews were conducted with clinic leadership separately. At the beginning of the interviews and focus groups, all participants received an IRB-approved information sheet and gave verbal consent to participate. The focus groups lasted approximately 60 minutes and the individual interviews 30 minutes. Participants were not remunerated for their participation. The interviews and focus groups were facilitated by two female clinical psychologists with experience conducting qualitative research (SY, KD) who had no prior relationship with the participants. The focus groups and leadership interviews were audio recorded, and a paid service transcribed the recordings verbatim. The software NVivo 12 was used for data management (QSR International, 2018).

Data Management and Analyses

The coding team was comprised of two bachelor’s level research assistants (DW and EF) who were trained in qualitative analyses by two experienced qualitative researchers (SY and KD) for the purposes of this study. The coding team was supervised by a post-doctoral fellow in psychology (KD). The clinical psychologist who conducted the interviews and focus groups (SY) and a psychiatrist who is one of the Principal Investigators of the primary protocol (AY) served as auditors for the coding process (Hill, 2012).

The data coding process included a directed content analytic approach (Hsieh & Shannon, 2005), whereby the coding team used an iterative process to analyze the de-identified transcripts starting with a codebook based on the CFIR constructs, and the themes that emerged were edited and adapted based on emerging data until saturation was achieved. All transcripts were coded independently by the two coders, who met with the supervising postdoctoral fellow on a weekly basis to discuss rationales for coding decisions as well as any discrepancies. Level of agreement was examined for all transcripts and consensus was reached through discussion between the two coders and the supervising post-doctoral fellow. After each transcript was coded, the results were sent to the auditors, who further reviewed the results to ensure that the coders remained consistent over time with their codes as well as to minimize the effects of groupthink (Hill, 2012). Consensus between the auditors and the coding team was also reached through discussion, until all transcripts were coded using the most updated version of the codebook.

Results

Study participants included 31 clinicians, 11 clinic leaders, and 8 administrators recruited from the four behavioral health clinics (Table 2 for participant demographics). Table 3 summarizes identified barriers and facilitators to implementing MBC by stakeholder group within identified CFIR constructs. Stakeholders did not provide any responses coded as the following CFIR constructs: intervention source, cost, peer pressure, self-efficacy, individual stage of change, and other personal attributes.

Table 2.

Individual and Focus Group Participant Demographics

Demographics	Administrators (N=8)*	Clinicians (N=31)**	Leadership (N=11)***
	N (%)
Sex (% male)	0 (0.0)	13 (41.9)	3 (30.0)
Hispanic	2 (25.0)	0 (0.0)	0 (0.0)
Race
White	4 (50.0)	21 (67.7)	9 (90.0)
Black/ African American	1 (12.50)	2 (6.5)	0 (0.0)
Asian	2 (25.0)	3 (9.7)	1 (10.0)
Multiple races	(0.0)	1 (3.2)	0 (0.0)
	Mean (SD)
Age, in years	35.5 (13.1)	41.0 (10.2)	52.6 (9.6)
Years in practice	9.5 (6.0)	10.1 (9.2)	21.8 (10.8)
Years at current clinic	5.1 (5.4)	5.6 (6.3)	17.8 (11.8)

^*Missing data on race for 1 participant

^**Missing data on sex, race, and years at current clinic for 4 participants; missing data on age and years in practice for 5 participants

^***Missing demographic data for 1 participant

Table 3.

Facilitators and barriers to the implementation of measurement-based care in behavioral health clinics identified by administrators (A), clinicians (C), and leadership (L) within Consolidated Framework of Implementation Research (CFIR) constructs.

	Stakeholder group
	A	C	L
Intervention Characteristics
Evidence Strength and Quality		+/−
Relative Advantage	+	+/−	+
Adaptability	+	+	+
Trialability			+/−
Complexity	−	+/−	+/−
Design Quality & Packaging	+/−	+/−	+/−
Outer Setting
Patient Needs & Resources	+/−	+/−	+/−
Cosmopolitanism			−
External Policies & Incentives		−	+/−
Inner Setting
Structural Characteristics	−	+/−	+/−
Networks & Communication	−	+/−	+/−
Culture	−	+/−	+/−
Implementation Climate	+/−	+/−	+/−
Readiness for Implementation	+/−	+/−	+/−
Characteristics of Individuals
Knowledge & Beliefs about the Intervention	−	+/−	+/−
Individual Stage of Change		+

(+)=facilitator, (−)=barrier, blank cells indicate a facilitator or barrier was not identified for that CFIR construct by a stakeholder group.

CFIR domains where stakeholders agreed upon the presence of a barrier or facilitator but focused on different aspects of MBC implementation within the barrier or facilitator are summarized below. Representative quotes in the text have been edited for clarity and the full quotes can be found in Supplement B.

Intervention Characteristics.

Relative advantage:

Stakeholders viewed using PROMs to enhance clinical care as a relative advantage over existing clinical practice as a facilitator in different ways. Leadership explained that the use of PROMs would be most beneficial in helping clinicians collect information for certain aspects of their clinical work where they may have had less specialty training (such as working with children or assessing substance use): “…if we had…this validated measure of…broadband …psychopathology, it could be really helpful in aiding…supervision to…identify areas that…clinicians need more support with and then for quality improvement,” L1. Clinicians and administrators highlighted that having the PROMs results available before an appointment would help the clinician be more focused in their session: “…That way we’re able to see… [the PROMs] and…use the interview…in a more sophisticated way to…draw upon that data…,” C7. Administrators also stated that PROMs would ensure that no important clinical questions are missed during the clinician’s encounter: “…Previously I think they asked all these questions. Or maybe some weren’t, so it might be that’s why we have this,” A5.

Adaptability:

All stakeholders agreed that if there were a system that could be adapted to a clinic’s specific needs surrounding the administration of PROMs, that this would facilitate MBC implementation. Administrators reported it would be helpful to choose during what type of appointment PROMs would be administered to decrease the volume of PROMs being administered daily across the clinic, and leadership similarly identified PROMs administration at certain appointments as important in helping administrators track when to administer PROMs: “…If there’s a way to just narrow it down either by visit type of provider, this way it’s less patients that are receiving it per day, so we don’t have to worry about either 50-plus, 300-plus patients that we either have to track or we have to give them the PROMs,” A3. Clinicians reported that it would be helpful to be able to adjust the frequency of PROMs administration as clinically indicated based on the type of treatment and clinical symptoms: “…I also see a lot of people for short-term CBT who I only see for three or four months. So, it would be nice to have the data gathered a little more frequently,” C29. Lastly, clinicians also noted that it would be helpful if patients completed PROMs before the appointment so that they do not need to spend time collecting this information themselves. “…it would be easier for the patient to fill out some kind of questionnaire, and then walk in with those kinds of things. So I don’t need to spend my time…collecting information type of things,” C8.

Complexity:

All stakeholders identified barriers to the MBC system. Leadership reported it is difficult for clinicians to find the data collected through PROMs in the EHR: ‘They can’t find the information. They tell us this over and over again. “It’s almost useless to me,”‘ L8. Clinicians reported they did not know how to interpret scores from the PROMs questionnaires, and that it was difficult to review the data longitudinally because this involved additional steps which were not clear: “And the other thing is I had to find my own, go online and figure out what the numbers meant,” C23. Administrators reported they struggled to know when to administer PROMs despite prompts to administer PROMs from the EHR because the questionnaires were administered at different frequencies, and patients sometimes reported they had recently completed the questionnaire: ‘…But there’s some questions that are every 30 days, and…some questions that are every 90 days. So, if the patient does one today, but the one from 90 days is already up in 10 days when they come back, they’re like, “I already did this”…,’ A3.

Design quality and packaging:

All stakeholders identified facilitators and barriers related to the design quality and packaging of MBC. To facilitate implementation, leadership and clinicians suggested patients complete questionnaires several days before the appointment, and leadership suggested patients could be prompted to do this through a patient portal: “…But can’t this be something that’s pushed out through Patient Gateway (patient portal) for patients to answer before they even get here?... that would be the best implementation and use of technology…,” L8. Leadership and clinicians also reported that data from the PROMs questionnaire need to be easy to quickly find within the EHR: “I don’t think you should have to go click ten places in order to find this. I think the purpose of it is for clinicians to be able to see it right when their note pops up,” C13, and easy to visualize over time: “I’d be very happy to see it over time. I’d like to see the graph for my patients,” L11. Clinicians also suggested information on how to interpret the questionnaire score be included for clinicians when they view the information within the medical record: “What’s the range for mild, and…moderate on this scale? There’s no additional information about what the numbers mean clinically,” C22. Administrators suggested the system should be designed to minimize redundancy in data collection when patients have multiple appointments on the same day: “…if the patients have multiple appointments in the same day, if they fill it out for one appointment, it goes away for the rest of the day… So, they don’t have to redo it again for another appointment…,” A3.

Outer Setting.

Patient needs and resources:

Stakeholders also identified some patient characteristics that may serve as facilitators in implementation. These included leadership’s awareness of the importance of patient education regarding PROMs questionnaires and how these will be integrated into treatment, “Patient-wise…they need to understand why, understand how it connects, and…feel like it was looked at,” L9. Stakeholders also described the need for flexibility regarding when the questionnaire is completed (before, during, or after an appointment). Administrators noted that it might be easier to introduce PROMs to new patients: “It’s also a lot easier to get a first-time patient…to do it than somebody who [comes in]…once every three months,” A1. Clinicians reported PROMs questionnaires could help identify important treatment goals, such as quality of life and social determinants of health (e.g., housing), that are often not directly addressed as part of clinical care: “…But one of the things we often are not so effective helping people with is…helping people get on with the business of living and rebuilding their shattered lives…,” C9.

Inner Setting.

Structural characteristics:

All stakeholders identified specific structural characteristics that served as barriers. Clinicians and administrative staff highlighted that the decentralized leadership and silo-ed nature of clinics lead to differing staff workflows and difficult implementation efforts: “I think that’s part of the problem that there’s so many players. Even a small innovation requires multiple teams of people in a place like this to really get on the same page,” C9. Leadership pointed out various changes occurring at the clinics, such as changes in front desk staff workflows and physical space moves would be added challenges. “We’re co-locating with a different clinic that has a different workflow, a slightly different philosophy, caring for people with slightly different needs, and being in the same space. And so, there’s going to be lots of adjustments,” L4. Clinicians and leadership both highlighted the lack of built-in time into sessions or workflows for PROMs administration and review as a barrier to implementation: “Unlike some clinics that have rooming and wait times sort of built in—you don’t expect to see your doctor right at 2 o’clock…Here, they kind of do, and they arrive anywhere between 1:58 and 2:07…that doesn’t leave you a lot of time,” L4. Administrators further added that the large patient caseloads make it even more difficult to make changes to workflows. “Well…we have an extreme patient volume, so…there’s difficulty in giving PROMs out, because the patient doesn’t come early. And by the time we get the line worked out…, it’s difficult to give the tablet to them,” A5.

Networks and Communications:

All stakeholders reported barriers related to communication across all levels of stakeholders. Leadership talked about the lack of formal channels for new ideas as a barrier in implementation, “…So I would say there is close to zero process [laughter]. But the ideas come from multitudes of people…,” L9, whereas administrative staff and clinicians focused on the challenges in disseminating new initiative information to patients and other providers in other clinics respectively as a barrier: “And some [clinicians] don’t even know about it, like when we’re doing it, right? The PROMs, like what is that for? And they don’t know that it’s in their chart every time they do it. So, more communication…will help,” A7.

Culture:

All stakeholders agreed on key cultural aspects that may pose as barriers for the intervention implementation. Clinicians and administrative staff noted that some siloes were hesitant to embrace new ideas possibly due to lack of leadership buy-in, leading to a disjointed atmosphere: “And I think with lots…modifications and adjustments to how we do things clinically,…getting everybody united in adopting the same new improvements, buying in and remembering to apply them in any kind of clinical interaction I think it’s tough as well,” C7. Clinicians also added that depending on clinicians’ backgrounds and roles, they may be less likely to embrace using PROMs as part of clinical care (i.e., dependent on their previous training, less likely to interact with EHR as part of routine care), and leadership also agreed that some clinicians believe that their focus of treatment is either SUD or psychiatric symptoms, and do not find screening for the other helpful: “…there has been this ongoing hesitation to engage with mental health treatment and substance use treatment at the same time. People feel like, “Oh, I can’t touch depression because what’s the point, until they’re sober?” Or vice versa, like, “Wow. You wouldn’t be depressed if you—” or, “You wouldn’t be drinking if you weren’t depressed…,” L4. Clinicians and leadership both agreed that the clinic atmospheres are too busy to implement PROMs or any new ideas, and leadership stated that clinicians are more likely to prioritize session time over PROMs completion if patients are late. Leadership stated that there is a “culture of business” in the clinic, which results in clinicians and administrative staff being asked to participate in new initiatives, regardless of whether they are helpful clinically or can be added as part of the workflow: “I think ideas are embraced quickly…and not always implemented very well, so the results in broad departmental improvements…is fairly minimal…,” L9.

Implementation Climate:

Regarding barriers, clinicians and leadership highlighted how logistical challenges, such as differing clinical workflows, can decrease clinicians’ PROMs receptivity: “…Psychologists don’t, as part of their workflow, use the computer as much. So, if they were going to give patients feedback or refer to the screening tools, if they’re not engaging with the computer, they won’t know to do that…,” L4. Leadership also noted less clinician receptivity to substance use PROMs, “[Psycho]pathology, I suspect people will look at and feel okay about. SUD, you’re now adding another layer of crap. Now I have to do something with something I don’t want to do anything about,” L10. Both leadership and administrative staff anticipated differential patient-level receptivity of the measures due to the measures’ length, confidentiality issues, and clinical utility. Administrative staff stated the importance of having clinic-level alignment when it comes to new initiatives in order to enhance systematic adoption of PROMs: “…we have experienced in the past [in] doing PROMs that the clinicians aren’t on board, frankly. They excuse their patients from doing it often, so then…what’s the point, and it’s a waste of our time. So…if we don’t have the clinician support, then it doesn’t go anywhere,” A1.

All stakeholders emphasized the importance of administrators as key facilitators in the process of integrating PROMs into the clinics, especially for sustainability purposes. Administrators focused on the importance of having clinicians’ review and follow up on PROMs results as a way to increase patient receptivity: ‘Like [the providers] get into their appointment, notice the PROMs wasn’t completed and they say, “Where’s your PROMs?” So that there’s some responsibility on the patient… They know it’s an expectation from their doctor,’ A1. Clinicians and leadership also stated that the clinical utility of PROMs, especially for depression and anxiety, from the providers’ and patients’ perspectives could facilitate implementation.

Readiness for Implementation:

All stakeholders identified both barriers and facilitators impacting implementation in this construct. There was consensus among stakeholders in the importance of having administrators be an integral part and resource in the implementation of PROMs at the clinics, but only administrators added that lack of training in the specific content of PROMs makes it difficult for them to help with the implementation: “I also feel like we don’t know what to say when we’re giving it to them…Because I’m not sure what it’s for exactly,” A7.

All stakeholders identified that the lack of time and physical clinic space, lack of additional personnel, and lack of clear workflows both in administering PROMs and provider review of results would serve as barriers in implementation. Clinicians specified lack of clinician buy-in with MBC as a challenge in implementation: “But the clinic has to be behind it. You have to educate people in health screen trainings, staff meeting. I mean, it has to be a coordinated effort to get people to use it,” C16.

All stakeholders identified specific steps that could facilitate implementation. Administrators focused on changes to clinic policies, such as having patients arrive 10–15 minutes before their appointment to complete PROMs or sending patients PROMs through the patient portal prior to their appointment, as facilitators: “…It says on one of their intake packets to come 10 or 15 minutes early…to complete any paperwork that might be needed” A1. Clinicians also suggested education for providers on the benefits of using PROMs: “The other thing I think the leadership could do is lead, tell us what it is, why it’s here, what it means, how to get it..,” C18. Both clinicians and leadership highlighted the importance of having champions that serve as the go-to person for PROMs related tasks and questions, including training clinicians on how to access PROMs and use the results for clinical purposes, addressing logistics related issues and training administrators and clinicians on these changes: “…And so, if you have the more senior staff who are…on board and championing this, then [new staff will] be like, “Okay. Well, this must be important. I need to be doing this as part of my job…,” L1. Leadership and clinicians noted streamlining the PROMs system, having technical support and integration with the existing EHR system, and tangible steps in implementation would be facilitators as they would decrease burden on administrators.

Characteristics of Individuals.

Knowledge and beliefs:

All stakeholders identified prior beliefs about PROMs and clinician and/or administrator lack of knowledge about PROMs as barriers. Both leadership and clinicians stated that providers not knowing the clinical applicability of PROMs was a barrier in their adoption, with some clinicians even reporting that they did not find administering PROMs to be clinically relevant for them, or that their experience has been that patients will not complete measures: “But hardly anyone actually finishes it. Even with a short amount of questions…,” C11. Clinicians expressed mixed opinions on using PROMs information to inform transfer of care: “I could kind of say, “Hey, look, you’ve been doing great. Your scores look great. It’s time to transfer back to your primary care doctor now,” C24, and “…it’s sort of like, well, what is the endpoint? Is it you haven’t used in a month, time to go, or are you still using?” C27. Administrators’ lack of knowledge of PROMs’ clinical usage, administration to patients via the EHR system, and lack of knowledge about the content of the PROMs questions were identified as barriers: “I don’t know how PHQ-9s are administered, if those are something that’s administered, or the doctor just does it on their own. I’m not sure, but they’re trying to push that and the something-7. QA-something?,” A1.

Discussion

The current study examined the similarities and differences regarding what multiple stakeholders, including clinicians, clinic leaders, and administrative staff, identified as barriers and facilitators to the implementation of MBC. Overall, the results show that there was convergence among stakeholders regarding which CFIR implementation constructs serve as barriers and facilitators, with all stakeholders identifying a barrier and/or facilitator for eleven of the sixteen constructs identified. However, there were also thematic differences that emerged amongst the stakeholders in similarly identified barriers and facilitators within implementation constructs. The thematic differences identified and prioritized within these implementation constructs aligned with their role in MBC.

Clinicians for example, were focused on the impact of MBC on clinical care including the validity and accuracy of patient reported outcomes. They also emphasized that they had limited time to incorporate MBC into their practice. Thus, data collection, interpretation, and tracking over time needed to be efficient, and the information collected easy to access. Our findings are consistent with other qualitative studies of behavioral health clinicians which have also focused on the importance of measure validity and MBC integration into clinical care (Garland et al., 2003; Meehan et al., 2006; Sharples et al., 2017). These findings are also consistent with the clinician’s role within MBC which includes interpreting PROMs data and discussing the information with the patient to inform the patient’s treatment plan. However, there was one clinician related theme identified in the literature that did not emerge in our study. Several qualitative studies of behavioral health clinicians identified concern regarding how PROMs data would be used by leadership at the healthcare system level including apprehension that this information could be used to restrict care or used to assess clinician effectiveness (Meehan et al., 2006; Sharples et al., 2017; Wolpert, 2014). This theme may not have emerged in our work since systematic PROMs collection was already an established practice within the larger healthcare system where our study took place, and the behavioral health clinics had some prior experience with a core component of MBC, PROMs administration and collection (Table 1).

In contrast to clinicians, clinic leaders were uniquely focused on overall clinic operations, and how patients and clinicians would respond to MBC implementation. Clinic leader comments also reflected concern regarding burdens and challenges that clinicians and administrative staff were already managing. These findings are consistent with qualitative research examining barriers and facilitators to implementing a new behavioral health intervention. When compared to patients and clinicians, clinic leadership was more focused on implementation barriers related to clinic operations (Youn et al., 2019). Our findings are consistent with the clinic leader’s responsibility for overall clinic operations, which include not only the patient’s experience in treatment, but also as staff satisfaction and staffing (clinicians and administrative staff).

Lastly, administrative staff were focused on patient flow through clinic, the administration and logistics of PROMs, and the patient’s experience when completing measures. This is also consistent with their MBC role to facilitate PROMs administration. To our knowledge input from frontline administrative staff has been absent when evaluating MBC implementation in the behavioral health setting as the current literature reports solely on provider and leadership perspectives. The administrative staff role focuses on supporting patient engagement in care (scheduling, checking in patients for appointments, insurance, or questionnaire collection). As such, it is essential to include administrative staff as part of the team when developing and implementing clinical practices that involve them to ensure successful implementation.

Since organizational alignment has been identified as a key factor in the success or failure of implementation initiatives (Lundmark et al., 2021), the results of our study emphasize the importance of understanding, addressing, and aligning the unique perspectives of each of the stakeholders. All stakeholder groups identified barriers/facilitators within the same CFIR implementation constructs. However, the differences between stakeholder groups in terms of the specific identified barriers/facilitators highlights this need to deliberately assess and address alignment as part of implementation. The identified barriers/facilitators may prove useful in conceptualizing system-wide changes to facilitate communication between stakeholder groups, and tailor trainings that match each stakeholder’s role in the MBC process. Developing distinct trainings for key stakeholder that matches their role during implementation has also been identified in the literature as a key implementation strategy when implementing a new practice(Edmunds et al., 2013; Sisodia et al., 2020).

Our findings are informative but also need to be considered in the context of this study’s limitations. These include the fact that participants were recruited from behavioral clinics that all had some experience with MBC, and they may have been biased by their past experiences with MBC. These clinics are also part of a well-funded healthcare system that has centralized infrastructure to support MBC, and that largely serves a privately insured patient population less impacted by social determinants of health including literacy, which may affect the generalizability of these results. Furthermore, data were also collected prior to the COVID-19 pandemic when outpatient behavioral health services were largely in person and PROMs administration was planned to be in person. Behavioral health care now commonly occurs through telehealth which necessitates alternative workflows for PROMs administration. Nonetheless the study highlights different perspectives which are applicable to in person PROMs administration and reflected all steps in the multistep MBC process that are still relevant.

Despite these limitations, the current study emphasizes the importance of collecting barriers and facilitators data from multiple stakeholders involved in the implementation of MBC in behavioral health. The perspective of administrators has been underrepresented in the literature despite their prominent role in the MBC process. The current study highlights their distinct perspective in the behavioral health setting. The omitted information that arises from not having the nuanced perspectives that contribute to barriers may result in poor adoption, low saturation, and lack of long-term sustainability. Having all stakeholders align in the barriers and strategies to address these aids enhances organizational alignment around the implementation of MBC and thus, its successful adoption (Lyon et al., 2018).