Skip to main content
Journal of General Internal Medicine logoLink to Journal of General Internal Medicine
. 2019 Dec 2;35(4):1001–1010. doi: 10.1007/s11606-019-05537-y

Time and Organizational Cost for Facilitating Implementation of Primary Care Mental Health Integration

Mona J Ritchie 1,2,, JoAnn E Kirchner 1,2, James C Townsend 1,3, Jeffery A Pitcock 1, Katherine M Dollar 4, Chuan-Fen Liu 5,6
PMCID: PMC7174254  PMID: 31792866

Abstract

Background

Integrating mental health services into primary care settings is complex and challenging. Although facilitation strategies have successfully supported implementation of primary care mental health integration and other complex innovations, we know little about the time required or its cost.

Objective

To examine the time and organizational cost of facilitating implementation of primary care mental health integration.

Design

Descriptive analysis.

Participants

One expert external facilitator and two internal regional facilitators who helped healthcare system stakeholders, e.g., leaders, managers, clinicians, and non-clinical staff, implement primary care mental health integration at eight clinics.

Intervention

Implementation facilitation tailored to the needs and resources of the setting and its stakeholders.

Main Measures

We documented facilitators’ and stakeholders’ time and types of activities using a structured spreadsheet collected from facilitators on a weekly basis. We obtained travel costs and salary information. We conducted descriptive analysis of time data and estimated organizational cost.

Key Results

The external facilitator devoted 263 h (0.09 FTE), including travel, across all 8 clinics over 28 months. Internal facilitator time varied across networks (1792 h versus 1169 h), as well as clinics. Stakeholder participation time was similar across networks (1280.6 versus 1363.4 person hours) but the number of stakeholders varied (133 versus 199 stakeholders). The organizational cost of providing implementation facilitation also varied across networks ($263,490 versus $258,127). Stakeholder participation accounted for 35% of the cost of facilitation activities in one network and 47% of the cost in the other.

Conclusions

Although facilitation can improve implementation of primary care mental health integration, it requires substantial organizational investments that may vary by site and implementation effort. Furthermore, the cost of using an external expert to transfer facilitation skills and build capacity for implementation efforts appears to be minimal.

Electronic supplementary material

The online version of this article (10.1007/s11606-019-05537-y) contains supplementary material, which is available to authorized users.

KEY WORDS: facilitation, implementation research, integrated primary care, mental health, primary care

INTRODUCTION

Integrating mental health services into primary care settings can improve access to mental health care, clinical outcomes, and cost efficiency for patients with mental and behavioral health conditions [13]. Such programs are called by various names, e.g., integrated primary care [4], primary care behavioral health [5], integrated care [6], collaborative care management [7], and primary care mental health integration [8, 9]. For this paper, we selected the latter because it is the term utilized in the context in which this study was conducted. Models of primary care mental health integration (PCMHI) are complex and challenging to implement, requiring change in the structure and processes of care delivery [2, 6, 10, 11]; provider values, attitudes, roles, and skills [1, 3, 6, 12, 13]; and organizational culture [6, 14, 15], including ways of working together [6, 12]. Further, primary care settings and their capacity for change vary widely [1, 1419]. Thus, such programs and the implementation process must be tailored to the needs and resources of the organizational context and PCMHI stakeholders, e.g., leaders, providers, and other staff [10, 17].

Implementation facilitation, a multi-faceted process of interactive problem-solving and support conducted by a designated individual (a facilitator) [20, 21], has successfully supported implementation of complex innovations and care delivery models such as PCMHI [11, 2226]. To address implementation challenges, facilitators apply a broad range of strategies, including engaging and building relationships with key stakeholders to obtain buy-in and participation, fostering team development, providing education and training, and clarifying stakeholder roles and responsibilities [21, 2729]. Facilitators also help and enable stakeholders to assess current practices, needs, and resources; plan implementation; monitor implementation progress; conduct ongoing problem identification and resolution; and improve care delivery [20, 21, 26].

Given the complexity of PCMHI programs and potential barriers to implementing them, clinical settings are often unable to implement PCMHI without assistance. Facilitation strategies, particularly those whose costs can be leveraged across more than one practice, are an attractive strategy for providing such assistance. Only two studies, however, have reported facilitation costs and neither addressed PCMHI implementation [30, 31]. Additionally, no studies have accounted for organizational costs associated with stakeholder participation time. This study provides decision-makers with a realistic assessment of the costs of facilitation.

We conducted a large mixed methods project testing a blend of external and internal facilitation within the context of Department of Veterans Affairs (VA) efforts to establish evidence-based PCMHI, including Collaborative Care Management and Co-located Collaborative Care, system-wide [32]. Results showed that clinics receiving facilitation had significantly higher PCMHI reach and adoption, as well as improved program uptake, quality, and adherence to evidence [25, 33]. This article describes the time and effort facilitators and VA stakeholders, i.e., leaders, clinicians, and other staff, devoted to facilitation activities and the organizational cost of facilitation for eight VA primary care clinics. We also sought to understand how facilitators’ time varied across settings, types of activities, and over time.

METHODS

This descriptive study of facilitation time and cost was conducted from August 2009 through November 2011. The VA Central Institutional Review Board approved and monitored the conduct of the study.

Study Clinics

To select sites that would receive facilitation, we first selected two VA networks (A and C) based on strength of mental health service line structure, current efforts to support PCMHI implementation, and ability to identify an internal facilitator at 50% effort. Network A had stronger mental health service line structure, operational authority, and existing network level support for PCMHI than network C. The mental health leader in each network identified four primary care clinics that planned to implement PCMHI but would require assistance and served 5000 or more patients. For the evaluation of facilitation’s effectiveness, reported elsewhere, we compared these facilitation clinics to matched clinics in two other networks (B and D) [25, 33]. However, this study focuses exclusively on the eight clinics in networks A and C that received facilitation.

Clinics receiving facilitation varied in size, serving from 4715 to 34,805 primary care patients. Because VA Medical Centers (VAMCs) have administrative responsibility for primary care clinics located at the VAMC and outpatient clinics located elsewhere, facilitation activities with VAMC leaders and managers may influence implementation at any of their clinics. Six of our clinics were under the administration of three VAMCs. For purposes of description and analysis, we grouped the clinics that were under the administration of a common VAMC as paired clinics, including three pairs: A1/A2, A3/A4, and C1/C2. Two clinics (C3 and C4) were under the administration of two different VAMCs; each was measured independently (see Table 1 for additional clinic characteristics).

Table 1.

Characteristics of Study Clinics

Clinic Clinic size (no. of PC patients) No. of PC providers Location Academic affiliation Administration
Network A
  A1 VAMC PC 5632 6 Midwest/urban Yes Clinics A1 and A2 under the administration of a common parent VAMC
  A2 CBOC 9224 12 Midwest/urban Yes
  A3 CBOC 4025 6 Midwest/urban Yes Clinics A3 and A4 under the administration of a common parent VAMC
  A4 CBOC 5654 6 Midwest/urban No
Network C
  C1 VAMC PC 34,805 16 Northeast/urban Yes Clinics C1 and C2 under the administration of a common parent VAMC
  C2 CBOC 14,763 12.6 Northeast/urban Yes
  C3 CBOC 8125 8 Northeast/urban No Clinic C3 under the administration of a different parent VAMC
  C4 CBOC 4715 5 Northeast/urban No Clinic C4 under the administration of a different parent VAMC

VAMC VA Medical Center, PC primary care, CBOC community-based outpatient clinic

Implementation Facilitation Strategy

The implementation facilitation strategy consisted of an external facilitator (EF), who worked with and mentored two internal regional facilitators (IRFs), one in each network, to support implementation of evidence-based PCMHI programs. The EF was a national expert in implementation facilitation, PCMHI, and implementation science. The two IRFs, a psychologist and a social worker experienced in PCMHI practices but novice to facilitation, were hired by their network mental health leaders. Although the study supported 50% of IRFs’ and facilitation support staff salaries, our cost estimate accounted for the actual time spent on facilitation activities and did not include time spent on research, e.g., documenting time, or non-study-related activities.

External and internal facilitators helped VA personnel across levels of the healthcare system identify staff who were stakeholders in PCMHI and its implementation. Thus, stakeholders varied by setting. Across settings, facilitators engaged key leaders and managers, PCMHI providers and program managers, primary care providers, nurses, and other staff identified as important. Utilizing a mix of in-person and virtual facilitation, facilitators worked with stakeholders, both individually and in groups, across phases of implementation. Facilitators assessed local contexts and practices; provided academic detailing and education about PCMHI and implementation processes; helped stakeholders design, apply, monitor, and evaluate implementation plans; addressed implementation barriers; and adapted PCMHI to local needs and resources while fostering adherence to evidence-based care delivery models. They also helped train PCMHI providers, established regional learning collaboratives, and helped stakeholders integrate PCMHI into organizational systems and processes. Detailed information about facilitation activities has been previously reported [25, 33] and summarized in Appendix 1.

Measures and Data Collection

To prepare for documenting facilitation time, MJR/JEK, based on facilitators’ experiences in a pilot study [34], identified types of facilitation activities: assessment, education and marketing, network development, preparation and planning, problem identification and resolution, program adaptation, stakeholder engagement, technical support, IRF training, and travel (see definitions in Appendix 1).

Facilitators documented their activity and travel time on structured spreadsheets (see Appendix 2), collected weekly for the duration of the study, a total of 28 months, generating a total of 1957 records. For each record, facilitators documented time, types of activities, sites that received their services, and the primary activity accounting for most of the time. They excluded research activities. They designated each record (spreadsheet row) as either a “single event,” defined as a meeting or activity of 15 min or more duration, or a “summary” of events, which included multiple activities, each of short duration. Each of the single-event records also listed individual stakeholders who participated in facilitation activities. JCT/MJR reviewed facilitators’ spreadsheets weekly. When records included multiple sites, to avoid overcounting, MJR determined how the time should be allocated by applying a set of rules (see Appendix 3).

To estimate personnel costs, we obtained annual salaries for facilitators and stakeholders through publicly available Federal Salary Web Portals [35, 36]. Salary costs were calculated as the estimated hourly rate multiplied by the time spent on facilitation activities. Cost estimates included the salaries of two network-level facilitation support staff who, according to IRFs, assisted at 25% effort each. Personnel costs included 30% for fringe benefits. We did not account for inflation because the inflation rate was zero during this time period.

Facilitators’ travel expenses included actual travel costs for the EF and estimated IRFs’ costs using General Services Administration mileage and per diem reimbursement rates.

Data Analysis

We calculated the hours spent by facilitators (EF and IRFs) in support of PCMHI implementation as well as the number of persons (person counts) and hours (person hours) invested by participating stakeholders. We conducted a separate descriptive analysis of travel cost, including salary for travel time and travel expenses.

RESULTS

Facilitator and Stakeholder Time Spent on Facilitation Activities

Table 2 summarizes EF and IRF activity and travel time for each network. The EF devoted 141 h in network A and 122 h in network C, approximately 0.05 FTE (excluding travel time), to train IRFs and support implementation at all 8 clinics during the study. The network A IRF (IRF-A) spent more time on facilitation activities (1792 h) than the network C IRF (IRF-C) (1169 h). Travel accounted for a significant proportion of EF time (29% in network A and 50% in network C); including travel, the EF devoted approximately 0.09 FTE to facilitation efforts. Travel only accounted for a small portion of IRF time in both networks.

Table 2.

Facilitator Activity and Travel Time Across Networks

Facilitators Network A Network C
Hours (%) Hours (%)
External facilitator (EF)
  Facilitation activities 141 (71) 122 (50)
  Travel 57 (29) 121 (50)
    EF Total 198 (100) 243 (100)
Internal regional facilitator (IRF)
  Facilitation activities 1792 (91) 1169 (81)
  Travel 171 (9) 273 (19)
    IRF total 1963 (100) 1442 (100)
Total 2161 1685

Stakeholders across organizational levels participated in facilitation activities (Table 3). In total, 133 and 199 stakeholders participated in networks A and C, respectively. Network A had more network level stakeholders (20 versus 10) and fewer clinic (54 versus 92) and VAMC (46 versus 85) stakeholders than network C.

Table 3.

Number of Stakeholders and Their Time Spent by Network

Type of stakeholder Network A Network C
Person counts (%) Person hours (%) Person counts (%) Person hours (%)
Clinic stakeholders 54 (41) 564.5 (44) 92 (46) 571.0 (42)
  Key leaders/managersa 8 72.0 7 105.5
  PCMHI providersb 10 343.8 12 269.5
  PC providersc 13 70.2 22 75.5
  PC nursesd 6 24.0 10 19.0
  MH specialty providerse 15 47.5 10 31.5
  All othersf 2 7.0 31 70.0
VAMC stakeholder 46 (35) 352.0 (27) 85 (43) 669.3 (49)
  Key leadersa 13 112.5 8 168.8
  Clinical/operational managers 5 21.5 15 70.8
  PCMHI managers/providersb 4 112.3 6 133.3
  Clinical staffg 12 43.2 41 248.0
  Non-clinical staff 12 62.5 15 48.4
Network stakeholder 20 (15) 302.1 (24) 10 (5) 69.8 (5)
  Key leadersa 3 74.7 3 30.5
  Program leaders 5 75.5 4 22.5
  Non-clinical staff 6 28.5 2 15.8
  Consultants 6 123.4 1 1.0
National leaders and other experts 13 (10) 62.0 (5) 12 (6) 53.3 (4)
Total all personnel 133 (100) 1280.6 (100) 199 (100) 1363.4 (100)

PCMHI primary care mental health integration, PC primary care, MH mental health, VAMC VA Medical Center

aKey leaders and managers including directors/associate directors, chiefs of staff, PC and MH leaders or care line managers, nurse managers, and clinic managers

bPCMHI providers included social workers, psychologists, psychiatrists, and nurses who provided mental health services. At the VAMC level, there were also PCMHI program managers

cPC providers include MDs, DOs, and nurse practitioners

dNurses include both registered nurses and licensed practical nurses

eMH specialty providers include psychiatrists, psychologists, social workers, registered nurses, and addiction therapists

fAll others at the clinic level included program managers, other professional staff, and support staff who were minimally involved in facilitation activities

gPC and MH providers, nurses, and PCMHI providers; in network C, 88% of these were located at VAMCs administratively responsible for but not including study clinics

In total, stakeholders contributed 1280.6 person hours in network A and 1363.4 person hours in network C. In network A, clinic stakeholders devoted the highest number of person hours (564.5), followed by VAMC (352.0) and network (302.0) stakeholders. In network C, clinic and VAMC stakeholders accounted for most of the person hours (571.0 and 669.3, respectively) (see Appendix 4 for more detailed information).

Organizational Cost of Implementation Facilitation

The total organizational costs of the implementation facilitation strategy during the 28-month period, including salary support for the EF, IRFs, facilitation support staff, and stakeholders, as well as travel expenses, were $263,490 in network A and $258,127 in network C. Salary support for IRF-A facilitation activities was higher than IRF-C ($100,193 versus $65,763). Network C ($97,975) had a higher stakeholder participation cost than network A ($81,418) (see Fig. 1). The organizational cost of salary support for facililtation activities, excluding travel salary support and expenses, was $236,263 in network A and $208,314 in network C.

Figure 1.

Figure 1

Organizational cost of implementation facilitation. EF external facilitator, IRF internal regional facilitator, VA Department of Veterans Affairs, VAMC VA Medical Center.

Variation in Facilitators’ Time Across Settings, Types of Activities, and Over Time

The EF and IRFs did not devote equal amounts of time across clinics within each of the networks. For example, in network A, the EF and IRF-A spent relatively more time helping clinic A2 (22%) compared to other clinics. In network C, the EF devoted more time to clinic C2 (19%), while IRF-C devoted more time to clinic C3 (20%). Additionally, IRF-C devoted more time (56%), compared to IRF-A (24%), on activities intended to benefit all clinics in a network (see Table 4 for a detailed comparison of facilitators’ time by network and by clinic).

Table 4.

Facilitator Time Across Clinics and Regional Networks

Participating clinics EF IRF Total
Hours (%) Hours (%) Hours
Network A
  A1 VAMC PC clinic 16 (11) 318 (18) 334 (17)
  A2 CBOC 25 (18) 403 (23) 428 (22)
  Both clinics A1 and A2 9 (7) 33 (2) 42 (2)
    Total: clinics A1 and A2 50 (36) 754 (42) 805 (42)
  A3 CBOC 16 (11) 305 (17) 321 (17)
  A4 CBOC 19 (13) 281 (16) 300 (16)
  Both clinics A3 and A4 4 (3) 17 (1) 20 (1)
    Total: clinics A3 and A4 38 (27) 602 (34) 641 (33)
  Across network A 52 (37) 436 (24) 488 (25)
  Total—network A 141 (100) 1792 (100) 1933 (100)
Network C
  C1 VAMC PC clinic 16 (13) 64 (5) 80 (6)
  C2 CBOC 23 (19) 101 (9) 125 (10)
  Both clinics C1 and C2 2 (2) 4 (< 1) 6 (< 1)
    Total: clinics C1 and C2 42 (34) 169 (14) 210 (16)
  C3 CBOC 12 (10) 230 (20) 241 (19)
  C4 CBOC 20 (17) 111 (9) 131 (10)
  Across network C 48 (39) 660 (56) 708 (55)
  Total—network C 121 (100) 1169 (100) 1290 (100)

Across network A and across network C indicate time spent in facilitation activities with or intended to benefit all of the clinics in network A or network C, e.g., learning collaborative calls for clinics’ PCMHI providers

EF external facilitator, IRF internal regional facilitator, VAMC Department of Veterans Affairs medical center, PC primary care, CBOC community-based outpatient clinic

The results show variation across facilitators in the proportion of time each spent on particular primary activity categories (Fig. 2). In both networks, the EF spent proportionally more time on problem identification and resolution (24% in network A and 19% in network C) and preparation and planning (16% in network A and 23% in network C) than on other activities. Compared to network C, the EF devoted relatively more time in network A to problem identification and resolution (24% versus 19%), stakeholder engagement (14% versus 8%), and training and mentoring IRF-A (13% versus 6%) and less time on assessment (6% versus 16%) and preparation and planning activities (16% versus 23%).

Figure 2.

Figure 2

Facilitator hours by primary activity. The charts above illustrate the hours and percentages of time facilitators spent on primary activities in each network. a illustrates the external facilitators’ hours and percentages of time on facilitation activities in networks A and C; b illustrates the internal regional facilitators’ hours and percentages of time.

Both IRFs spent the most time on preparation and planning as a primary activity (43% for IRF-A and 45% for IRF-C), and they were working alone for most of this time. IRF-A spent proportionally more time, compared to IRF-C, on stakeholder engagement (12% versus 4%) and problem identification and resolution (12% versus 7%) and proportionally less time on IRF training and mentoring (5% versus 15%).

Figure 3 displays the number of hours spent by each facilitator by month over the course of the study. The EF’s time in both networks was relatively stable, tapering off in the last 4 months. Time spent by IRF-A was also relatively stable compared to IRF-C. Both IRFs were continuing to provide support at the end of the study.

Figure 3.

Figure 3

External facilitator and internal regional facilitator time by month. EF external facilitator, IRF internal regional facilitator.

DISCUSSION

This is the first study to assess the time, effort, and organizational cost of facilitating PCMHI implementation. Additionally, it is the first study of facilitation cost to include stakeholders participating in implementation facilitation activities, which is vital for implementation success [37, 38]. Our study found that clinics which needed assistance to implement PCMHI may require substantial organizational investment in resources. We also found that time and cost varied across settings, types of activities, and over time. Study findings provide important insights for planning and applying facilitation strategies.

The substantial amount of resources utilized to facilitate implementation, ranging from $208,314 to $236,263 per network excluding travel cost, is not surprising given the complexity of PCMHI [1, 6, 13], the difficulty of changing the structure and processes of care [15, 39, 40], the large number of contextual barriers that can hinder implementation [10, 12, 15, 16, 19], and the number of stakeholders that might need to be involved. Our study findings support the need to protect time for stakeholder participation in facilitation efforts. Furthermore, significant IRF time was dedicated to implementation throughout the study period, suggesting that some of the clinics needed ongoing assistance for an extended period of time. Based on our data, intensive facilitation strategies like this one can be costly and are likely not required for all sites to successfully implement PCMHI. Future research should explore how to identify sites likely to benefit from facilitation and the dose and duration of facilitation needed to address implementation barriers.

Variation in the type and amount of resources utilized across settings have implications for clinical leaders, managers, and researchers planning to apply facilitation strategies [41]. For example, although our previously reported findings suggested that clinics C1, C2, and C4 had more implementation challenges [33] and therefore might need more assistance, this study shows that IRF-C devoted more time to clinic C3, which was under the administration of the facility where she was located. Perhaps convenience or leadership expectations influenced IRF-C’s use of time. Harvey and colleagues suggest that ongoing monitoring of facilitator time, activities, and fidelity to the facilitation model may increase attention to sites with the highest need and ultimately implementation outcomes [42]. Unfortunately, in this study, the EF could not monitor IRFs’ time. Further exploration of how best to transfer facilitation knowledge and skills from external to internal facilitators as well as tools to monitor fidelity to the implementation facilitation model are needed.

Additionally, IRF-C devoted considerably less time than IRF-A to facilitation efforts, possibly due to competing demands and or differences in career stage, e.g., IRF-C was preparing for retirement and IRF-A was early in her career. On the other hand, IRF-C spent relatively more time on activities intended to benefit all four clinics in the network, e.g., organizing network-wide educational meetings. This was consistent with her previous experiences but was likely also related to contextual factors, e.g., network C lacked infrastructure support for PCMHI implementation present in network A at the start of the study. Our findings suggest that personal factors should be considered when selecting facilitators, and those to whom facilitators are accountable need to protect time for facilitation efforts, ensure that facilitation is provided based on need for assistance, and match facilitation strategies to organizational need as well as facilitator skills.

We encountered several unexpected findings. First, the time and cost of external facilitation, which included mentoring IRFs, were surprisingly minimal. This finding suggests that the facilitation strategy is an efficient model for transferring facilitation skills to develop novice facilitators into experts, thus building implementation capacity for future initiatives. Future research should explore methods and techniques for transferring these skills. Second, although facilitation is a highly interactive process [21, 27, 34], IRFs documented the largest proportion of their time across the length of the study on preparation and planning activities. It is likely that facilitators spent some of this time developing processes and tools that could be applied in subsequent implementation efforts, thus representing an initial time investment that would decrease preparation and planning time in future facilitation efforts. Therefore, there is potential for an “economy of scale” as facilitation is spread to additional sites, which should be explored in future studies.

This study has several limitations. First, we explored time and cost from the perspective of only two inter-related cases (the EF and IRF dyad in two regional networks). Second, we conducted the study in eight clinics in a large US integrated healthcare system for a veteran population. However, the facilitation strategy is applicable to a broad range of healthcare settings within and outside of this country [3]. The final limitation is related to the challenges of collecting time data, including the burden of documentation and the potential for inconsistencies when more than one individual documents time. [43, 44] To reduce burden, the facilitators summarized their time and/or documented multiple activities on some records, which may have resulted in underreporting stakeholder time and facilitation time spent on activities that were not identified as primary activities. Therefore, our time and cost estimates may be lower bound estimates. To ensure data quality, we conducted a rigorous review process and ongoing training. Despite the limitations, our findings provide a baseline for future studies and our study methods are being adapted and applied in other facilitation studies [45].

In summary, the study facilitation strategy, designed to address both the complexity of PCMHI programs and implementation challenges in primary care settings, requires substantial organizational investments that may vary by clinic, healthcare system, stakeholder involvement, and facilitation approach. However, the transfer of facilitation skills from an external expert to initially novice facilitators inside their healthcare systems required minimal resources and built capacity for future implementation efforts. Recognizing its value, VA national mental health leadership adopted this facilitation strategy to support implementation of PCMHI and other initiatives in facilities across VA [32]. Our findings provide useful information to organizations, researchers, and policy makers seeking to implement PCMHI, particularly in clinical settings needing additional implementation assistance.

Electronic supplementary material

ESM 1 (498.2KB, docx)

(DOCX 498 kb)

Funding Information

This study was funded by the U.S. Department of Veterans Affairs, Quality Enhancement Research Initiative (QUERI), Service Directed Project #08-316, JoAnn Kirchner, PI. The funding body had no involvement in designing the study; collecting, analyzing, and interpreting data; or writing the manuscript. The views expressed in this article are those of the authors and do not represent the views of the U.S. Department of Veterans Affairs or the United States Government.

Compliance with Ethical Standards

The VA Central Institutional Review Board approved and monitored the conduct of the study.

Conflict of Interest

The authors declare that they do not have a conflict of interest.

Footnotes

Prior Presentations

4th Biennial Society for Implementation Research Collaboration Conference, Seattle, WA, September 9, 2017

AcademyHealth 2017 Annual Research Meeting, New Orleans, LA, June 25, 2017

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

References

  • 1.Butler M, Kane RL, McAlpine D, et al. Integration of Mental Health/Substance Abuse and Primary Care. Evidence Report/Technology Assessment No. 173 (Prepared by the Minnesota Evidence-based Practice Center under Contract No. 290-02-0009). Rockville, MD: Agency for Healthcare Research and Quality 2008. Report No.: 09-E003.
  • 2.Kwan BM, Nease DE. The state of the evidence for integrated behavioral health in primary care. In: Talen MR, Valeras AB, editors. Integrated behavioral health in primary care. New York: Springer; 2013. pp. 65–98. [Google Scholar]
  • 3.WHO, Wonca. Integrating mental health into primary care: a global perspective. Geneva (Switzerland): World Health Organization and World Organization of Family Doctors (Wonca) 2008.
  • 4.Martin MP, White MB, Hodgson JL, Lamson AL, Irons TG. Integrated primary care: a systematic review of program characteristics. Fam Syst Health. 2014;32(1):101–15. doi: 10.1037/fsh0000017. [DOI] [PubMed] [Google Scholar]
  • 5.Beehler GP, Lilienthal KR, Possemato K, et al. Narrative review of provider behavior in primary care behavioral health: how process data can inform quality improvement. Fam Syst Health. 2017;35(3):257–70. doi: 10.1037/fsh0000263. [DOI] [PubMed] [Google Scholar]
  • 6.Davis M, Balasubramanian BA, Waller E, Miller BF, Green LA, Cohen DJ. Integrating behavioral and physical health care in the real world: early lessons from Advancing Care Together. J Am Board Fam Med. 2013;26(5):588–602. doi: 10.3122/jabfm.2013.05.130028. [DOI] [PubMed] [Google Scholar]
  • 7.Liu CF, Hedrick SC, Chaney EF, et al. Cost-effectiveness of collaborative care depression treatment in a primary care veteran population. Psychiatr Serv. 2003;54(5):698–704. doi: 10.1176/appi.ps.54.5.698. [DOI] [PubMed] [Google Scholar]
  • 8.Kehle SM, Greer N, Rutks I, Wilt T. Interventions to improve Veterans’ access to care: a systematic review of the literature. J Gen Intern Med. 2011;26(2):689. doi: 10.1007/s11606-011-1849-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Chang E, Rose D, Yano E, et al. Determinants of readiness for primary care-mental health integration (PC-MHI) in the VA Health Care System. J Gen Intern Med. 2013;28(3):353–62. doi: 10.1007/s11606-012-2217-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Cohen DJ, Balasubramanian BA, Davis M, et al. Understanding care integration from the ground up: five organizing constructs that shape integrated practices. J Am Board Fam Med. 2015;28(Suppl 1):S7–S20. doi: 10.3122/jabfm.2015.S1.150050. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Solberg LI, Crain AL, Jaeckels N, et al. The DIAMOND initiative: implementing collaborative care for depression in 75 primary care clinics. Implement Sci. 2013;8:135. doi: 10.1186/1748-5908-8-135. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Hall J, Cohen DJ, Davis M, et al. Preparing the workforce for behavioral health and primary care integration. J Am Board Fam Med. 2015;28(Suppl 1):S41–S51. doi: 10.3122/jabfm.2015.S1.150054. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Talen MR, Valeras AB, editors. Integrated behavioral health in primary care. New York: Springer; 2013. [Google Scholar]
  • 14.Maruthappu M, Hasan A, Zeltner T. Enablers and barriers in implementing integrated care. Health Systems & Reform. 2015;1(4):250–6. doi: 10.1080/23288604.2015.1077301. [DOI] [PubMed] [Google Scholar]
  • 15.Van den Broeck K, Ketterer F, Remmen R, Vanmeerbeek M, Destoop M, Dom G. Why collaborative care for depressed patients is so difficult: a Belgian qualitative study. Int J Integr Care. 2017;17(2):7. doi: 10.5334/ijic.2491. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Whitebird RR, Solberg LI, Margolis KL, Asche SE, Trangle MA, Wineman AP. Barriers to improving primary care of depression: perspectives of medical group leaders. Qual Health Res. 2013;23(6):805–14. doi: 10.1177/1049732313482399. [DOI] [PubMed] [Google Scholar]
  • 17.Patel V, Belkin GS, Chockalingam A, Cooper J, Saxena S, Unützer J. Grand challenges: Integrating mental health services into priority health care platforms. PLoS Med. 2013;10(5):e1001448. doi: 10.1371/journal.pmed.1001448. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Tomoaia-Cotisel A, Scammon DL, Waitzman NJ, et al. Context matters: the experience of 14 research teams in systematically reporting contextual factors important for practice change. Ann Fam Med. 2013;11(Suppl 1):S115–S23. doi: 10.1370/afm.1549. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Lau R, Stevenson F, Ong BN, et al. Achieving change in primary care-causes of the evidence to practice gap: systematic reviews of reviews. Implement Sci. 2016;11:40. doi: 10.1186/s13012-016-0396-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Stetler CB, Legro MW, Rycroft-Malone J, et al. Role of “external facilitation” in implementation of research findings: a qualitative evaluation of facilitation experiences in the Veterans Health Administration. Implement Sci. 2006;1:23. doi: 10.1186/1748-5908-1-23. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Dogherty EJ, Harrison MB, Graham ID. Facilitation as a role and process in achieving evidence-based practice in nursing: a focused review of concept and meaning. Worldviews Evid Based Nurs. 2010;7(2):76–89. doi: 10.1111/j.1741-6787.2010.00186.x. [DOI] [PubMed] [Google Scholar]
  • 22.Alagoz E, Chih M-Y, Hitchcock M, Brown R, Quanbeck A. The use of external change agents to promote quality improvement and organizational change in healthcare organizations: a systematic review. BMC Health Serv Res. 2018;18(1):42. doi: 10.1186/s12913-018-2856-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Wang A, Pollack T, Kadziel LA, et al. Impact of practice facilitation in primary care on chronic disease care processes and outcomes: a systematic review. J Gen Intern Med. 2018. 10.1007/s11606-018-4581-9 [DOI] [PMC free article] [PubMed]
  • 24.Baskerville NB, Liddy C, Hogg W. Systematic review and meta-analysis of practice facilitation within primary care settings. Ann Fam Med. 2012;10(1):63–74. doi: 10.1370/afm.1312. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Kirchner JE, Ritchie MJ, Pitcock JA, Parker LE, Curran GM, Fortney JC. Outcomes of a partnered facilitation strategy to implement primary care-mental health. J Gen Intern Med. 2014;29(Suppl 4):904–12. doi: 10.1007/s11606-014-3027-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Roderick SS, Burdette N, Hurwitz D, Yeracaris P. Integrated behavioral health practice facilitation in patient centered medical homes: A promising application. Fam Syst Health. 2017;35(2):227–37. doi: 10.1037/fsh0000273. [DOI] [PubMed] [Google Scholar]
  • 27.Lessard S, Bareil C, Lalonde L, et al. External facilitators and interprofessional facilitation teams: a qualitative study of their roles in supporting practice change. Implement Sci. 2016;11(1):1-12. 10.1186/s13012-016-0458-7 [DOI] [PMC free article] [PubMed]
  • 28.Bidassie B, Williams LS, Woodward-Hagg H, Matthias MS, Damush TM. Key components of external facilitation in an acute stroke quality improvement collaborative in the Veterans Health Administration. Implement Sci. 2015;10(1):69. doi: 10.1186/s13012-015-0252-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Berta W, Cranley L, Dearing JW, Dogherty EJ, Squires JE, Estabrooks CA. Why (we think) facilitation works: insights from organizational learning theory. Implement Sci. 2015;10:141. doi: 10.1186/s13012-015-0323-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Hogg W, Baskerville N, Lemelin J. Cost savings associated with improving appropriate and reducing inappropriate preventive care: cost-consequences analysis. BMC Health Serv Res. 2005;5:20. doi: 10.1186/1472-6963-5-20. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Culler SD, Parchman ML, Lozano-Romero R, et al. Cost estimates for operating a primary care practice facilitation program. Ann Fam Med. 2013;11(3):207–11. doi: 10.1370/afm.1496. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Pomerantz AS, Kearney LK, Wray LO, Post EP, McCarthy JF. Mental health services in the medical home in the Department of Veterans Affairs: factors for successful integration. Psychol Serv. 2014;11(3):243–53. doi: 10.1037/a0036638. [DOI] [PubMed] [Google Scholar]
  • 33.Ritchie MJ, Parker LE, Kirchner JE. Using implementation facilitation to foster clinical practice quality and adherence to evidence in challenged settings: a qualitative study. BMC Health Serv Res. 2017;17:294. doi: 10.1186/s12913-017-2217-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Kirchner J, Edlund CN, Henderson K, Daily L, Parker LE, Fortney JC. Using a multi-level approach to implement a primary care mental health (PCMH) program. Fam Syst Health. 2010;28(2):161–74. doi: 10.1037/a0020250. [DOI] [PubMed] [Google Scholar]
  • 35.DATA UNIVERSE: Federal Employee [database on the Internet]. Asbury Park Press. 2007 - 2010. Available from: http://php.app.com/agent/federalemployees/search. Accessed 12 Jun 2012
  • 36.FedsDataCenter.com: Search Federal Employee Salaries [database on the Internet]. FedSmith Inc. 2010 - 2011. Available from:https://www.fedsdatacenter.com/federal-pay-rates. Accessed 12 Jun 2012
  • 37.Parker LE, de Pillis E, Altschuler A, Rubenstein LV, Meredith LS. Balancing participation and expertise: a comparison of locally and centrally managed health care quality improvement within primary care practices. Qual Health Res. 2007;17(9):1268–79. doi: 10.1177/1049732307307447. [DOI] [PubMed] [Google Scholar]
  • 38.Norris JM, White DE, Nowell L, Mrklas K, Stelfox HT. How do stakeholders from multiple hierarchical levels of a large provincial health system define engagement? A qualitative study. Implement Sci. 2017;12:98. doi: 10.1186/s13012-017-0625-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39.Ion A, Sunderji N, Jansz G, Ghavam-Rassoul A. Understanding integrated mental health care in ‘real-world’ primary care settings: what matters to health care providers and clients for evaluation and improvement? Fam Syst Health. 2017;35(3):271–82. doi: 10.1037/fsh0000294. [DOI] [PubMed] [Google Scholar]
  • 40.Collins C, Hewson DL, Munger R, Wade T. Evolving models of behavioral health integration in primary care. Milbank Memorial Fund: New York; 2010. [Google Scholar]
  • 41.Saldana L, Chamberlain P, Bradford WD, Campbell M, Landsverk J. The Cost of Implementing New Strategies (COINS): a method for mapping implementation resources using the stages of implementation completion. Child Youth Serv Rev. 2014;39:177–82. doi: 10.1016/j.childyouth.2013.10.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42.Harvey G, McCormack B, Kitson A, Lynch E, Titchen A. Designing and implementing two facilitation interventions within the ‘Facilitating Implementation of Research Evidence (FIRE)’ study: a qualitative analysis from an external facilitators’ perspective. Implement Sci. 2018;13(1):141. doi: 10.1186/s13012-018-0812-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43.Lopetegui M, Yen PY, Lai A, Jeffries J, Embi P, Payne P. Time motion studies in healthcare: what are we talking about? J Biomed Inform. 2014;49:292–9. doi: 10.1016/j.jbi.2014.02.017. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44.Smith MW, Barnett PG. Direct measurement of health care costs. Med Care Rev. 2003;60(Suppl 3):74S–91S. doi: 10.1177/1077558703257001. [DOI] [PubMed] [Google Scholar]
  • 45.Bauer MS, Miller C, Kim B, et al. Partnering with health system operations leadership to develop a controlled implementation trial. Implement Sci. 2016;11(1):22. doi: 10.1186/s13012-016-0385-7. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

ESM 1 (498.2KB, docx)

(DOCX 498 kb)


Articles from Journal of General Internal Medicine are provided here courtesy of Society of General Internal Medicine

RESOURCES