Implementing a Continuum of Evidence-Based Psychosocial Interventions for People With Severe Mental Illness: Part 2—Review of Critical Implementation Issues

Abstract

Objective:

In North America and internationally, efforts have been made to reduce the gaps between knowledge of psychosocial evidence-based practices (EBPs) and the delivery of such services in routine mental health practice. Part 2 of this review identifies key issues for stakeholders to consider when implementing comprehensive psychosocial EBPs for people with severe mental illness (SMI).

Method:

A rapid review of the literature was conducted. Searches were carried out in MEDLINE and PsycINFO for reports published between 1990 and 2012 using key words related to SMI, and psychosocial practices and implementation. The Consolidated Framework for Implementation Research (CFIR) was used to structure findings according to key domains and constructs known to influence the implementation process.

Results:

The CFIR allowed us to identify 17 issues reflecting more than 30 constructs of the framework that were viewed as influential to the process of implementing evidence-based psychosocial interventions for people with SMI. Issues arising at different levels of influence (intervention, individual, organizational, and system) and at all phases of the implementation process (planning, engagement, execution, and evaluation) were found to play important roles in implementation.

Conclusion:

The issues identified in this review should be taken into consideration by stakeholders when engaging in efforts to promote uptake of new psychosocial EBPs and to widen the range of effective psychosocial services available in routine mental health care.

In most industrialized countries, gaps exist between knowledge of effective psychosocial practices and the application of these practices in routine mental health services.^1–4 However, such gaps are difficult to close, and reports suggest that translating knowledge into practice can take years and even decades.^5,6 As such, mental health authorities and service providers are increasingly seeking to understand the strategies and factors that contribute to implementation success or failure in an effort to accelerate the change process.

This review aimed to take stock of the evidence related to the implementation of psychosocial EBPs for people with SMI and to support future efforts to implement a broader range of services than is currently available in most communities. Part 1 of our review⁷ described more than a dozen international, national, and regional initiatives carried out since 1990 that aimed to implement multiple psychosocial EBPs (Table 1) and identified the key implementation strategies used to promote uptake of these EBPs. In part 2, we identify critical issues for stakeholders to consider when embarking on a process of implementing a broader array of psychosocial EBPs in their own settings.

Table 1.

Description of major initiatives aiming to implement multiple psychosocial EBPs for people with SMI

Initiative	Description and objectives	References
National Implementing Evidence-Based Practices Project United States (8 states): 1999–2006	Multi-site demonstration project aiming to support the implementation of 5 psychosocial interventions (ACT, family psychoeducation, illness management and recovery, integrated dual disorders treatment, and supported employment) in community mental health settings.	Bond et al16 Torrey et al50,66 Mueser et al55
Enhancing Quality-of-care In Psychosis United States (4 VHA service regions): 2001–ongoing	Project emerging from the VHA Quality Enhancement Research Initiative, which aimed to promote the delivery of guideline-concordant care (including family psychoeducation, weight management services, and supported employment) to veterans with schizophrenia.	Brown et al27
VHA Quality Improvement initiatives United States (across the United States): 2004–ongoing	Broad quality improvement initiatives led by the VHA to transform mental health services provided to veterans and provide a full continuum of recovery-oriented and psychosocial rehabilitation services.	McHugh and Barlow43 Goldberg and Resnick78
NIDA–SAMHSA Blending Initiative United States (across the United States): 2001–ongoing	Partnership between NIDA and SAMHSA aiming to accelerate implementation of research findings from NIDA-sponsored treatment studies into mental health and substance use services.	Martino et al41 Condon et al79
Systematic Treatment Enhancement Program for Bipolar Disorder United States (15 institutions across the United States): 1998–2005	A national, longitudinal infrastructure for clinical trials comparing the effectiveness of 4 psychosocial EBPs (CBT, family focused therapy, interpersonal and social rhythm therapy, and patient education with illness self-management) for people with various presentations of bipolar disorder.	Miklowitz and Otto22 Sachs et al64 Bowden et al80
Mental Health Treatment Study United States (19 states): 2006–2010	Large study that aimed to evaluate and improve employment policies and services for people with SMI receiving Social Security Disability Insurance.	Frey et al28,81
Housing First—At Home/Chez soi United States (across the United States): 1993–ongoing Canada (5 cities): 2009–ongoing	Gradual dissemination of supported housing intervention aiming to provide housing and treatment services to homeless people with SMI. At Home/Chez soi project conducted in Canada, the largest pragmatic trial of the Housing First model to date.	Pathways to Housing Inc32 Goering et al82
Optimal Treatment Project International: 1994–mid-2000s	An international, multi-site pragmatic trial evaluating the effectiveness of evidence-based biomedical and psychosocial treatments (including family psychoeducation, stress management training, ACT, social skills training, CBT, and early intervention programs) for people with psychosis.	Falloon56 Falloon et al83
DH mental health reforms United Kingdom (across England): late 1990s–ongoing	Mental health system reforms initiated by the British government and overseen by the DH. Reforms included implementation of several psychosocial supports for people with SMI (early intervention programs for psychosis, assertive outreach, and crisis services).	UK DH63,84,85 Joseph and Birchwood86
Reforms to psychiatric services in Israel Israel: 2000–ongoing	Legislation-based reforms broadening access to evidence-based psychiatric rehabilitation services carried out by the Israeli government and initiated by a multi-stakeholder group of mental health activists.	Roe et al61,87

CBT = cognitive-behavioural therapy; DH = Department of Health; NIDA = National Institute on Drug Abuse; SAMHSA = Substance Abuse and Mental Health Administration; VHA = Veterans Health Administration

Method

Search and Selection Process

Full details of the rapid review methods are available in Menear and Briand.⁷ Briefly, we carried out comprehensive searches in MEDLINE and PsycINFO (January 1990 to March 2012) for English- and French-language literature that included terms for SMI, and psychosocial interventions and implementation. All sources combined, our search yielded 2816 reports. After screening (Figure 1), we then extracted data from the 107 selected articles, guided by a conceptual framework.

Figure 1.

Flow chart of articles included in the review

Clinical Implications

• During the past 2 decades, the evidence base for a range of psychosocial interventions has grown considerably.

• When seeking to implement these interventions, stakeholders must consider a wide range of issues arising at different levels of influence (intervention, individual, organizational, and system) and stages of the implementation process (planning, engagement, execution, and evaluation).

• Consideration of the factors that contribute to implementation success or failure is important to reduce gaps in services and ensure the availability of a continuum of effective and sustainable mental health services.

Limitations

• The streamlined methodology adopted for this review may have led to some relevant articles being omitted from our analysis.

• Consideration of the public policy context of mental health service delivery in the different countries contributing research for the review was limited.

Conceptual Framework

The CFIR was recently developed by Damschroder et al⁸ to provide an overarching typology of key domains known to influence the implementation process. These 5 domains are: characteristics of the intervention, the individuals involved in the intervention, the organization (inner setting), the outer setting, and the implementation process. The theoretical constructs identified from published implementation theories were identified for each domain, for a total of 39 constructs (eTable 2).

Results

In the sections that follow, we present some of the main implementation issues that were identified by authors across the review studies. When it was judged appropriate, constructs from the CFIR were discussed in tandem.

Characteristics of the Intervention

Evidence Strength and Quality

Authors have long argued that a main barrier to the uptake of psychosocial interventions was probably the evidence itself.^9–12 While the evidence base for many interventions has grown considerably during the past decades, even now some psychosocial practices are recognized as having stronger empirical support than others, and there is still much to learn about intervention mechanisms and outcomes for different patient populations.^9,10,13 Capturing the critical ingredients and mechanisms of psychosocial interventions is complex work, but it is a necessary step in the process of implementing and evaluating these interventions.^13–15

Complexity

The multidimensional nature of interventions influences rates of uptake, with interventions requiring more skill to deliver or involving more professionals or service components being more difficult to implement faithfully.^16–20 In the NEBPP, difficulties in implementing integrated treatment for mental health and substance use disorders was directly linked to the complexity of the intervention and its implementation.¹⁶ As with other EBPs (for example, family psychoeducation, illness management, and recovery), integrated mental health and substance use treatment interventions require a high degree of skill development and behavioural changes on the part of individual practitioners along with organizational and structural service changes. In contrast, interventions relying more heavily on organizational and structural service modifications (for example, supported employment and ACT) were more often implemented successfully.¹⁶ Managing the complexity of implementation may be a particular challenge when seeking to implement multiple psychosocial EBPs, though this may be achieved through sequential (compared with simultaneous) implementation of EBPs²¹ or through the introduction of new practices while simultaneously enhancing fidelity and quality of existing psychosocial services.²²

Adaptability and Relative Advantage

Decisions on which services to offer in a particular setting result from deliberation on the evidence supporting interventions and on the needs of service users, ethical and legal issues, local culture, and the preferences and values of the individuals or groups involved.^10,23 While some authors argue fervently in favour of high-fidelity implementation,^10,23 others defend the notion that adaptations to local context or populations are acceptable as long as interventions and outcomes are evaluated.^24–27 If providers perceive interventions to be too rigid or fail to appreciate their potential advantages, they are unlikely to be adopted.^9,23,25

Costs

The cost-effectiveness of psychosocial EBPs was considered an important issue by several authors.^{12,19,21,28–33} Community-based EBPs are widely considered more cost-effective than traditional services over the long term,^12,33 especially when they generate cost reductions in other areas (for example, inpatient services).^29,30 That said, some psychosocial EBPs are clearly more costly to implement than others, and high start-up costs for EBPs (for example, ACT) can act as a barrier to uptake.^21,34

Characteristics of the Individuals Involved in the Interventions

Knowledge, Competence, and Self-Efficacy

To take ownership of a practice and progressively develop a positive attitude toward change, service providers need information about EBPs, and they need support in the form of training and ongoing supervision.^{13,19,35–42} Training allows providers to evaluate their level of comfort with new psychosocial practices and fosters commitment to change.⁴⁰ It also allows them to develop practical expertise and the skills needed to deliver EBPs and to believe in their own abilities to meet new challenges. Many psychosocial EBPs require providers possess sophisticated skills, and without proper training many providers feel underprepared to adopt these practices.^2,17,18,43 Successful training requires a balance of didactic (for example, written materials and workshops) and interactive (for example, case review meetings, supervision, audit, or self-appraisal) training approaches.^42,43

Attitude and Individual Stage of Change

Though pressures to adopt new or unfamiliar practices can be anxiety-provoking, change is facilitated when providers can tolerate some uncertainty and when positive attitudes toward change are nurtured and encouraged.^3,18 Resistance to change is a theme often highlighted in the implementation literature and was noted as a key barrier in many of the articles retrieved in this review. Resistance may stem from misalignments between provider philosophies and those underpinning new practices, negative perceptions of interventions, or pressures to develop new habits or skills.^17,19 Further, busy work schedules and a lack of time to integrate new knowledge can discourage openness to change and the adoption of new, complex EBPs.^{19,36,37,44,45} To overcome such obstacles, leaders within organizations must establish supportive practice contexts, engage and negotiate with team members, and provide them with time to learn and experiment with new practices.³ Failure to put time and resources into such actions often leads providers to continue doing what is familiar and comfortable to them (inertia) even though more effective practices may exist.¹⁵

Personal Attributes—Values and Professional Identity

Attributes, such as providers’ values and professional identity, play important roles in the implementation of EBPs.^10,18,23,33 Some providers may defend more humanist positions and display skepticism toward new practices or standardization efforts seemingly insensitive to sociodemographic or clinical differences among service users.^10,23,44 Implementation efforts may arouse fears of losing professional autonomy,^23,46 and implementation success may depend on professionals revisiting their self-image and surrendering some of their autonomy.¹⁸

Characteristics of the Organization (Inner Setting)

Organizational Structure and Workforce

The fragmentation of services, frequent personnel turnover, and shortages of highly trained specialist therapists that characterize many mental health systems hinder the implementation of needed psychosocial services and the development of advanced professional skills.^{19,29,42,44,45,47,48} Organizations’ ability to train professionals and retain qualified personnel is critical for successful implementation efforts and in the delivery of high-quality services.^28,49 Large organizations with a larger pool of professionals are often able to replace personnel that leave with people with new skills and knowledge of best practices.¹⁵ However, larger organizations also tend to have organizational structures that are more rigid, which may impede innovation.³ Investing in the organization’s workforce and retaining qualified personnel is thus key to implementing comprehensive psychosocial services.

Organizational Culture

Organizations must feature supportive structural characteristics and foster a culture that facilitates the emergence and maintenance of EBPs.^16,24,50,51 Providers may not fully appreciate the benefits of an EBP if the basic philosophy, values, and norms espoused in their work environment encourage a different service approach.^20,37,50,52 A major finding of the NEBPP was that many implementation sites were not naturally predisposed to integrating science-based interventions or to practising according to recovery philosophies, a cultural reality that needed to be addressed prior to the actual implementation of EBPs.⁵⁰

Implementation Climate

Similar to organizational culture, an organization may exhibit a climate that, at a given point in time and for various reasons, is favourable to the adoption of psychosocial EBPs. In initiatives such as the NEBPP and the Enhancing Quality-of-care In Psychosis Project, researchers purposely aimed to influence implementation climates by securing support for initiatives from key stakeholders and by generating enthusiasm for EBPs through project kick-off events with participating sites.^17,27 Initial implementation of EBPs is also facilitated by organizational climates that promote learning and are open to change, which can be made possible through training or knowledge-sharing opportunities, systems of rewards for new practices, and time protected for reflexive thinking and experimenting with new practices.^{13,15,35,40,44} Organizations should also foresee a consolidation phase when implementing EBPs, that is, ensuring sustainability of newly introduced EBPs by formalizing mechanisms for supervising staff, monitoring service costs and identifying quality issues or problems, and ensuring that services continue to meet client needs.³

Readiness for Implementation

Several factors seem to make organizations more ready to implement psychosocial EBPs, such as strong leadership and an organizational commitment to change practices,^{13,16,36,37,48,50} the availability of adequate resources,^{2,15,17,21,40,46,53–55} as well as access to experts willing to support implementation and share their expertise.^16,28,32,56 When organizational leaders make practice change a clear priority, they can mobilize their personnel and foster a climate that overcomes implementation obstacles. This was apparent in the NEBPP, where sites with engaged leadership found ways to ensure they achieved high fidelity in the EBPs being implemented (for example, by hiring staff to fill specific roles, limiting the size of caseloads, organizing team meetings, and revising productivity standards that conflicted with the EBP).¹⁶ Strong leadership is even more critical for practices that involve important changes in philosophy, coordination with many stakeholders, or political and financial issues.^20,21

There is little doubt that inadequate funding and under-staffed or -skilled programs and services are all key impediments to the implementation of psychosocial EBPs.^{17,23,29,44,52,57} Lack of funds to pursue professional development and the absence of incentives to promote high-quality services can dampen staff morale and slow the uptake of EBPs.^{9,13,15,29,44,45} Timely access to people with expertise and perceived legitimacy with EBPs is another key facilitator of change. In the Optimal Treatment Project, the ongoing support and technical assistance of the project leader was identified as a major determinant of success in the implementation of best practices in the treatment of schizophrenia across numerous countries.⁵⁶

Outer Setting

Mental Health Authority Leadership and Engagement

As with organization-level leadership, leadership exhibited by mental health authorities and policy-makers is widely considered critical to the implementation of psychosocial EBPs.^{16,17,21,31,47–49,58,59} For instance, in the NEBPP, lack of commitment on behalf of mental health authorities in some states acted as a major barrier to the implementation and sustainability of psychosocial EBPs.^18,60 In contrast, support in other states from such authorities took the form of clear policies in favour of psychosocial EBPs, support in training and consultation (for example, through the establishment of technical assistance centres), and implementation of financial incentives and quality-improvement strategies (for example, through licensing and accreditation processes).^16,17 When mental health authorities played a lead role in implementation, they established norms for high-quality services, ensured that there were no geographic disparities in services, and acted as influential change agents.^21,59

Attitudes and Advocacy by Service Users and Families

While involving service users and their families in psychosocial treatment and service planning has historically been challenging, it is increasingly thought to facilitate the adoption of new practices and to decrease stigma.^13,18,44,57 Service users often have particular concerns, such as fears that services they appreciate and consider effective could be discontinued or replaced by unfamiliar services having stronger empirical support.¹¹ Further, advocacy by service users and families has been shown to accelerate the dissemination of EBPs and ensure that new practices correspond to real community needs.^11,12 A striking example is the case of psychiatric service reforms in Israel, where consumers, families, and other activists successfully pressured the government to significantly broaden the range of psychosocial EBPs available to people with SMI.⁶¹ In the past decade, the number of people receiving psychiatric rehabilitation services in Israel has quadrupled, and the budget for these services has increased 8-fold.⁶¹ Clearly, more collaborative approaches to implementation that are driven by the voices of consumers and families have great potential to increase access to psychosocial supports.

Implementation Process

Implementation Timelines

Implementing effective psychosocial practices can take considerable time, especially when multiple EBPs are implemented at once. While research reports may focus on the execution and evaluation phases of initiatives, the time required to plan initiatives and engage stakeholders in a new project should not be underestimated.^3,17,19,62 For instance, the Mental Health Treatment Study was carried out between 2006 and 2010, but was preceded by 6 years of conceptual development.²⁸ When timelines to implement EBPs are compressed, adequate communication and engagement with partners may not occur.²⁰

Collaboration and Support of Stakeholders

A common strategy adopted in implementation initiatives is to engage multiple stakeholders in the implementation process. Shared interest in new practices and concerted efforts to initiate change can facilitate implementation of EBPs.²⁰ In some initiatives, the creation of steering committees or implementation teams was used as a strategy to facilitate the involvement of key partners and to develop consensus around implementation plans and issues.^{16,20,27,63,64}

Skills of Front-Line Players

The literature is clear that administrative and mental health authority leadership is critical to implementation success, but the skills and determination of front-line program managers, clinical supervisors, as well as clinical advisers or consultants were equally found to be very important.^16,17,48,51 Front-line supervisors working with consultants must ensure that there is a commitment on behalf of partners based on target objectives, offer support and continuous feedback, including recommendations and concrete action plans, as well as rapidly resolve problems that emerge, including relational or political issues.^16,21,51 In the NEBPP, implementation suffered when clinical supervisors were unable to provide structure to providers’ practices and failed to master the skills needed for group supervision.⁵¹ Some supervisors further sabotaged the implementation process by failing to follow through with commitments they had made to the research team or to their own staff.⁵¹

Quality Monitoring and Evaluation

Many authors consider that regular evaluation of practice fidelity and quality is a major determinant of implementation success and improved outcomes for service users.^{2,10,14–16,25,28,41–43,48,65,66} Fidelity monitoring is thought to communicate key components of an intervention and progress with its implementation, document variation in practices across sites in multi-site studies, improve implementation by raising awareness of nonadherence to best practices, and predict client outcomes.²⁸ Where such evaluations were not valued or poorly executed, implementation typically suffered and services failed to meet the standards dictated by current evidence.¹⁶

Discussion

Our analysis, guided by the CFIR model, allowed us to shed light on a range of issues for stakeholders to consider when implementing psychosocial EBPs for people with SMI. Seventeen issues emerged as central to implementation efforts, reflecting more than 30 different constructs identified in the CFIR. In particular, the characteristics of interventions (for example, evidence, complexity, adaptability, and cost), providers (for example, knowledge, skills, attitudes, values, and identity), organizations (for example, culture, leadership, resources, and access to expertise and support), and the broader context (for example, community needs, policies, and incentives) were all found to have strong influences on practice implementation. In addition, critical implementation issues arise at various stages of the implementation process (for example, planning, engagement, execution, and evaluation).

Our review also allowed us to identify the range of players who must assume a collective leadership role to implement a continuum of psychosocial services for people with SMI. Successful implementation occurs when there is a synergy between stakeholders that emerges when each fulfils their necessary roles and commitments.⁵¹ The task of creating this synergy to implement single psychosocial EBPs in complex health systems is clearly not an easy one. The challenge, then, of ensuring that communities can access a broad continuum of psychosocial services can thus seem even more daunting. However, while the literature focuses largely on implementation issues related to single EBPs, the articles retained for our review provide some guidance for stakeholders seeking to establish a more comprehensive range of services.

First, implementation initiatives are facilitated when all key stakeholders are engaged in the process from the start, and communication channels exist for them to share their expertise and to voice their concerns with project leaders. As argued by Rosenheck,⁶² the strength of these coalitions and their ability to mobilize partners and promote change has certainly had as much or more of an influence on eventual outcomes than on the quality of the available evidence for the targeted EBPs. The active involvement of consumers and caregivers should be prioritized so services are evidence-based, correspond to their preferences, and reflect philosophies related to empowerment and recovery.^67–70 Several authors further argue that the influence of differences in resources or power held by stakeholders involved in the process should not be neglected or underestimated,^13,71,72 suggesting a need for engagement processes that are open, equitable, and empowering.²⁰

Second, efforts to educate providers about the value of evidence-based and recovery-oriented service approaches should precede or complement training strategies for specific EBPs. However, leaders should be careful to identify the learning needs of providers and introduce new psychosocial EBPs at a digestible rhythm so providers are not overburdened and their quality of working life does not suffer.^21,44,45 High staff turnover is common in the mental health sector, and work dissatisfaction and the precariousness of providers’ own mental health are serious problems.^16,73–75 Engaged leaders and organizational cultures of trust and learning can help providers build a sense of self-efficacy with new practices and ensure that these are aligned with their professional identity as well as the values established in the larger practice setting.

Third, practice contexts that embrace and support knowledge exchange, innovation, and evaluation clearly promote the emergence of new, effective practices more than contexts lacking the leadership, resources, external pressures, or expertise to overcome organizational inertia and alter the status quo. As many of the initiatives show, actions taken at multiple levels by numerous players can facilitate or impede the implementation of psychosocial services. Modern technologies and common systems to allow the widespread monitoring of services and client outcomes are urgently needed to support decision making and increase the quality and range of services for people with SMI.^31,48

Finally, authors maintain that establishing service continuums requires that stakeholders adopt a systems view of the broader organization of services, compared with a component view, a perspective that would allow them to appreciate how each component contributes to the whole, to detect gaps in service and training, and to promote creative and critical thinking about the manner in which the system functions.^3,76 Once such a perspective is achieved, action to effect changes and better address community needs depends on timely access to funding and resource management that promotes best practices.^12,23,48,77 In many jurisdictions, medical models guide the allocation of resources for mental health services, while budgets for community-based and psychosocial services remain limited and unprotected.^3,30,57

Conclusion

Modern mental health systems are currently striving to reduce gaps that exist between what is known to be effective and the services that are delivered in routine care. This task involves developing a strong understanding of the implementation process as well as the roles that actors at different levels must play to effectively bring about practice changes. Future efforts to implement psychosocial EBPs should solicit the engagement of all key stakeholders and adopt a systems perspective to reduce inequities in care and make accessible the broadest range of evidence-based services possible.

Abbreviations

ACT

assertive community treatment

CFIR

Consolidated Framework for Implementation Research

EBP

evidence-based practice

NEBPP

National Implementing Evidence-Based Practice Project

SMI

severe mental illness