Abstract
In mental health care, consumer providers (CPs) are individuals with serious mental illness (SMI) who draw upon their lived experiences while providing services to others with SMI. Implementation of CPs has proven to be challenging in a variety of settings. The PEER project (Peers Enhancing Recovery) involved rolling out CPs using an implementation science model and evaluating implementation and impact in mental health treatment settings (3 intervention, 3 control). In qualitative interviews, facilitators and challenges to implementation were described by the CPs, their team members, clients, and study researchers. Site preparation, external facilitation, and positive, reinforcing experiences with CPs facilitated implementation. Role definitions and deficiencies in CPs’ technical knowledge posed challenges to implementation. Sustainability was not realized due to insufficient resources. However, implementation was positive overall, characterized by Diffusion of Innovation concepts of high relative advantage, strong trialability, compatibility with prevailing norms, compelling observability, and relatively low complexity. By preparing and working systematically with intervention sites to incorporate new services, implementation was strengthened and challenges were minimized.
Keywords: consumer provider, mental illness, implementation, facilitation
Introduction
In mental healthcare, consumer providers (CPs) are individuals with serious mental illness (SMI) who draw upon their lived experiences of mental health disorders to provide services to others with SMI. Utilization of CPs is considered an innovative approach that can support clinical services in a number of ways. CPs can facilitate the learning of self-management skills, provide hope, engage clients into treatment, serve as role models, and help clients connect with natural supports in their communities. Consumer providers are working in a number of healthcare settings, and providing services such as Supported Employment, psychosocial weight management protocols, and intensive case management. There is growing evidence for the effectiveness of CPs in national and international mental health settings.1–5 However, those who have conducted studies of CPs have consistently noted challenges related to implementation of this role, including boundary issues, potential breaches in confidentiality, disclosure of disability status, discrimination, and poorly defined job structure.6–8 Furthermore, the nature of CP qualifications for employment (i.e., the requirement for personal experience with the clinical condition) necessitates delineation of professional behaviors in ways that might not be explicit for other workplace roles. Despite these implementation challenges, past studies of CPs have not used implementation science to facilitate the use of CPs in clinical settings.
The PEER project (Peers Enhancing Recovery) is the first study to use implementation science to guide the roll-out of CPs. The project was a one-year, cluster randomized controlled trial that compared three Mental Health Intensive Case Management (MHICM) teams which incorporated CPs, with three other MHICM teams that continued without CPs. Each intervention team was assigned two CPs. The study took place within the Veteran Health Administration (VHA). All six teams were at separate sites. However, all sites were within one Veterans Integrated Service Network (VISN) -- one geographic region of the US. The MHICM teams, modeled after Assertive Community Treatment,9 are designed to provide community-based, comprehensive management to individuals with SMI.
Implementation in the PEER project was guided by the Simpson Transfer Model,10,11 a model which draws on Diffusion of Innovation theory,12 and frames implementation of innovations as a staged process of organizational change from exposure to adoption, implementation, and practice (routine use). PEER implementation also built on lessons learned from previous exploratory studies of CP implementation.13,14 Given PEER’s novel use of implementation theories, one key goal was to evaluate facilitators and challenges to implementation of CPs on MHICM teams. This paper focuses on the implementation process from the perspective of the CPs themselves, their team members, clients, and study researchers, and thereby contributes to a growing literature informing implementation of innovative clinical programs in complex settings.11,15
Methods
Procedures
CPs were identified via application for the position on the federal jobs website. Those who applied were screened by Human Resources staff at the respective sites, the research team along with the MHICM staff interviewed the candidates put forward by Human Resources, and the best candidates were selected. CPs varied in terms of how much they chose to share about their personal experiences with mental illness and substance abuse; they were not required to fully disclose such experiences prior to being hired.
Prior to patient enrollment, all CPs received a 30-hour certification training on appropriate peer support, basic counseling skills, and psychosocial rehabilitation that was modeled after the Georgia model16 of Medicaid-reimbursed consumer providers that is now accepted by twenty-five states. CPs also received a two-day training on Illness Management and Recovery (IMR), an evidence-based, manualized curriculum aimed at helping clients with SMI learn skills to better manage their illness and achieve their goals.17
PEER research staff served as “external facilitators”18 (i.e., individuals who play a key role in helping teams understand what they need to change and how to change it19), helping the MHICM teams incorporate CPs into their operations. Guided by the Simpson Transfer Model, PEER research staff held information sessions about CPs (exposure), encouraged MHICM team leaders to hire the CPs (adoption), helped the MHICM teams tailor the CP positions to their teams (implementation), and provided weekly supervision to the CPs and met with MHICM team leaders monthly to troubleshoot any ongoing difficulties (practice). Full details of this facilitation can be found in Chinman, Shoai, & Cohen (2010).11 MHICM staff also provided weekly supervision and additional training on MHICM rules and procedures.
CPs conducted case management consistent with the MHICM model (which involves an assertive approach) with a focus on being supportive and encouraging, building hope, teaching skills, liaising to other providers, and using and describing their own experiences in the service of these tasks. They conducted IMR groups and individual sessions. They intended to participate in all MHICM activities (e.g., team meetings and events, charting in the medical record). There was some flux with CP employment, with some leaving earlier than expected (one due to illness, another due to enrolling in secondary education) and another transferring from one intervention team to another. No team of two CPs remained the same throughout the whole study.
All patients on each MHICM team’s caseload during the recruitment period were eligible for the study. Of the 404 who were eligible, 285 MHICM clients enrolled (69%), of which 149 were at intervention sites and 133 were at control sites. Of the 149 clients at intervention sites, 57% had direct contact (documented encounters) with the CPs. Client data collection took place from October 2006-May 2011. Client-level outcome assessments were conducted via survey before and after the one-year intervention and are described in Chinman et al.20 All procedures were approved by Institutional Review Boards at each site.
Implementation evaluation
MHICM teams consisted of a psychiatrist, psychologist, social worker, and nurse case manager (all of whom were considered non-CPs), and, on the intervention teams, two CPs. Each team had an administrator who oversaw the work of the whole team. To assess barriers and facilitators to CP implementation, individual interviews were conducted at each implementation site with providers (n=8) and CPs (n=5), and two focus group interviews were conducted with clients, who were randomly selected among those who had contact with CPs (n=8). Clients who could not attend the focus groups (n=2) completed individual interviews. Providers and CPs had agreed to participate in these interviews when the sites agreed to participate; there were no exclusion criteria. All staff, including CPs, who were asked to complete an interview agreed. Clients were paid $20 for their participation. Given their extensive role in facilitating implementation and supervising CPs, members of the research team (n=4) were also interviewed regarding their perceptions of implementation challenges and facilitators.
All individual interviews were conducted by the lead author, an experienced qualitative researcher, and focus groups were facilitated by [co-author] and [co-author]. Separate versions of the interview guide were developed for the following respondent types: providers who were MHICM administrators, other non-consumer providers, CPs, and clients. As depicted in Table 1, primary interview domains were guided by the STM in that we inquired about change over time (i.e., from exposure to the CP concept to adoption of the CP innovation, implementation, and practice/sustainability), and by DOI in that we investigated characteristics of the innovation during each stage of organizational change. Interviews lasted 30–60 minutes and were digitally recorded and professionally transcribed.
Table 1.
STM stage | DOI concept | Interview domains* |
---|---|---|
Exposure | Anticipated compatibility with organizational norms, anticipated complexity | Expectations of how CPs would “fit in” to the MHICM teams, what CPs would do, how clients would react to CPs, how difficult or easy it would be to integrate CPs |
Adoption | Relative advantage, trialability, compatibility, complexity | Early benefits of CPs and experiences of working with CPs; role of external facilitation in addressing challenges |
Implementation | Relative advantage, observability, compatibility, reinvention | Perceived advantages of CPs over time; CPs’ daily work activities; experiences of integration of CPs onto MHICM teams; site-specific adjustments made to facilitate implementation |
Practice/Sustainability | Relative advantage, compatibility, reinvention | Desire to maintain the CP role; value of the CP role; perceptions of the extent to which the CPs had become integrated onto the team; any changes that would be made to the CP role if maintained |
Adjusted to correspond to type of respondent
Data analysis
Transcripts were reviewed and edited for accuracy, and were then summarized by the lead author and discussed by the team, with a focus on consensus-building around the key domains (Table 1) and developing site-by-site comparisons. Based on these discussions and domains developed for the summaries, an initial top-level code list was developed and utilized via ATLAS.ti, a qualitative data software tool. Top-level codes were applied to broadly categorize responses within key domains of interest. Then each code’s content was reviewed and examined for emergent subcodes; additional codes were created when a theme was raised by the majority of respondents within a given role (e.g., non-consumer providers). Interviews were analyzed within and across sites, and within and across roles (using document families in ATLAS). Axial coding (i.e., combining of codes) was used to synthesize findings and reflect upon characteristics of the innovation over time. Coding was reviewed on a consistent basis by [co-author] and [co-author], and the team discussed and resolved differing perceptions as needed.
Results
Commonly described implementation facilitators included site preparation, external facilitation, and positive, reinforcing experiences with the CPs. Implementation challenges included role definitions and deficiencies in CPs’ technical knowledge. These facilitators and challenges, as well as sustainability issues, are described below.
Implementation facilitators
The research team’s knowledge of CP implementation and recovery-oriented MH services was critical in optimizing acceptance and utilization of CPs. Knowledge of challenges that had arisen in other CP initiatives in and outside VA was incorporated into intensive pre-implementation education with the MHICM teams and education and training of the CPs in order to prevent well-known problems from occurring, foster compatibility with organizational norms, and lessen perceived complexity of the new role. For example, the issue of confidentiality was a known concern of non-CP staff, i.e., there was an expectation that CPs would not maintain client privacy. A MHICM staff member noted the initial worry in the exposure stage about “boundary issues”:
There were different levels of openness to [CPs]. A lot of the things that we were really concerned about have really been sort of non-issues, which is funny. Like there was a lot of concern about the whole confidentiality issue, that they would have access to their peers’ private medical history and information and sort of the boundary issues. We were really worried about the boundary issues…Different people had different levels of comfort with it. There were definitely a lot of misgivings early on.
The issue of boundaries was thoroughly discussed during pre-implementation planning and subsequent supervision in order to help the CPs negotiate the balance between professionalism and empathy; CPs were explicitly instructed and reminded to convey to clients that their communications would be shared among the whole MHICM team. Staff across all three implementation sites referred to the importance of “laying the ground rules” for the CPs with regard to boundaries and patient confidentiality. CPs described their experiences of clients disclosing private matters that they did not want their providers to know, and their response of telling the MHICM staff about these incidents. CPs were clearly familiar with their mandate to share any and all information of clinical relevance. This preparatory work (and reinforcement via supervision and ongoing facilitation) with MHICM staff and CPs served to preclude challenges that had been encountered in previous efforts.
External facilitation by the research team was critical in supporting implementation. Non-CP staff, particularly MHICM administrators, appreciated having consistent access to expertise and resources, as well as being able to troubleshoot issues with the research team; this access seemed to mitigate the complexity of implementation. External facilitation was particularly important when staff within sites were having conflicts. For example, one site was undergoing a contentious management transition when the CPs were hired; one staff member noted that the CPs were “dropped into a hot zone” at this site. The research team was thereby in a position to facilitate the integration of the CPs (including serving as buffers) when internal staff dynamics were less than optimal. CPs also appreciated access to the research team, and benefited from weekly phone or in-person supervision, although some noted that they needed less supervision as they became more comfortable with their responsibilities and status within the team.
The demonstrated value of CPs over time facilitated better utilization of their services by clients (and more referrals from MHICM staff) over time; all staff conveyed that the CPs were a valuable addition to their teams, particularly when the CPs’ strengths and competencies had been identified and demonstrated. For example, a MHICM administrator observed the following about the effects of the IMR groups that the CPs had spearheaded:
I felt there was a significant treatment effect from watching clients who had previously been almost unable to maintain a certain level of social skills that, after several [IMR] group meetings, [clients were] more willing, more able, to maintain eye contact, to have appropriate interactions with those around them. They appeared to have relaxed significantly about some of their life dilemmas. I noticed a very significant positive effect from this treatment paradigm [CPs delivering IMR].
Staff remarked that the CPs were particularly helpful with the “more stable” clients who needed supportive visits, and that the CPs often had access to information from the clients that was not available to other staff:
[CPs are] privy sometimes to information that case managers aren’t, just based on their relationships with the vets [clients]. Sometimes people share things with them that they haven’t shared with us.
Staff also noted that CPs were advantageous not only to the clients but also to the MHICM teams. CPs did not tend to share their personal histories with their teams (though they did with the clients), but their presence and contribution were observed to “cut across professional elitism,” in the words of one MHICM staff member. Some MHICM staff felt that, in order to maintain professionalism, the CPs’ mental health histories should not be discussed with other staff members, despite the fact that these histories were “currency for the job.”
CPs themselves were also able to observe their own contributions, and this reinforced their sense of value on the teams and supported implementation. They noted that they served as role models to the clients. For example, one CP noted:
I’m proof that you can overcome. You can’t get completely healed…of your mental illness, ‘cause I still have it. But we can work it out where it’s not controlling us. Or we can control it, you know, keep tabs on it and live a normal healthy life. So now, because of that, [the clients] are more open with me now.
CPs also observed the importance of their empathic listening skills:
Sometimes when a veteran [client] has a problem, it won’t surface until the [CP] comes around. When he’s in the presence of the doctor, he can’t let the doctor know that he has this symptom. But when the [CP] is there, he sees someone that he can just go straight across with him. And he’ll open up and say, “Hey, I’m having this side effect.” And you can let him know because you’ve got experience. “Well, I had those side effects before, too. I heard voices, too.”
Eventually, I’d say most of the clients or veterans got to know me and started asking for me, because I’m not critical. I listen very well and I understand. I empathize.
Clients affirmed the importance of the CPs’ unique ability to listen:
The [CPs] take you out and try to get you [to discuss your problems]…instead of you just going to a psychiatrist. You can sit down a while and ask questions. Psychiatrists only ask questions.
You can really empathize from personal experiences what somebody who’s suffering mental illness, no book in the world, no film, no counseling, no facts is going to give it to you.
These beneficial experiences across non-CP and CP staff, as well as clients themselves, supported implementation even when challenges arose. For example, when a CP had an alcohol relapse, did not accept treatment, and eventually had to be fired (a common fear/expectation related to hiring those with histories of substance abuse), the event was addressed solely in the context of that one individual’s circumstances and not attributed to CPs as a whole. Further, when a second CP had a short alcohol relapse, she was given the opportunity to receive inpatient treatment and return to her job to continue to make positive contributions to the team. Both instances demonstrated that positive experiences with the other CPs far outweighed the challenges that arose.
Implementation challenges
Because implementation was positive overall, challenges rather than “barriers” (i.e., implementation was not precluded or halted by detrimental factors) were identified during the course of implementation. One implementation issue concerned which clients would be seen by the CP. MHICM teams assigned clients to the CPs differently, based on their assessment of each CP’s capacity. The two assignment approaches were: 1) having a set caseload for the CPs; or 2) wanting CPs to be flexible to see whichever client needed help on a particular day. MHICM administrators who favored the “assigned caseload” approach felt the CP would then know whom to contact and would have an opportunity to forge a bond over multiple visits and other interactions. CPs who were assigned a caseload of clients were typically those who were perceived to need “structure” and more intensive supervision. Administrators who favored the “flexible caseload” approach felt that it was responsive to the needs of a given day, or a given client on a given day. CPs who were allowed a flexible caseload were typically those who were perceived to be self-motivated and adaptable to emergent needs. A MHICM administrator noted,
[One CP] had individual cases that he was referred to by each case manager to kind of go out and see every week and work on them with their recovery goals. And then [the other CP] kind of just did whatever. She was very, very flexible. If she was asked to go and take a veteran to an appointment for that day and maybe do something later on, or if she had another veteran that she was already scheduled to see, she would make time in her schedule to do that, or if a veteran just needed a little bit of extra support for that day, she would go and she would offer that support, work on them with their recovery goals.
An unexpected challenge was that some CPs did not have sufficient knowledge of standard workplace behavior, or what one MHICM staff member called “basic work professionalism,” such as notifying the team when sick, arriving on time, responding in a timely period to messages, prioritizing tasks, and working proficiently with computers. Furthermore, CPs required more training than anticipated on documenting interactions with clients, particularly refraining from making clinical judgments based on their observations, an activity VHA had reserved solely for clinicians. Working with the VHA’s computerized medical record also posed a notable challenge. One staff member said, “I think the biggest problem was with [a CP who] didn’t have a clue to CPRS [Computerized Patient Record System].” Training CPs in workplace behavior and utilization of CPRS was time-consuming for some MHICM staff, particularly administrators. However, this challenge resolved over time, as the CPs became more aware of expectations and more familiar with their tasks.
Sustainability
Despite strong support and extensive efforts of the PEER research team, MHICM administrators, and other key stakeholders, CPs were not hired beyond the life of the project because of limited funding at the time (CPs could have been transitioned from study funds to clinical funds at their sites). Unfortunately this meant that what had become a valuable component of each team’s services and resources could not be maintained, and this was frustrating to non-CP and CP staff members alike. MHICM staff members across all of the sites expressed disappointment at the prospect (post-implementation) of losing the CPs, and noted that the impact of this loss would be felt by the staff and the clients alike.
Discussion
Implementation of CPs in three intensive case management teams was positive overall, after some “growing pains” were overcome. In this case, positive implementation can be operationalized by examining characteristics of the innovation—i.e., the CPs and the services that they provided—according to Diffusion of Innovation theory (DOI).12 After staff were exposed to the idea and the logistics of the CP role and CPs were trained in their duties, the CP role was adopted in the implementation sites. Once the tasks and responsibilities of the CPs were established in real time, their value became apparent, and their contributions were perceived by staff and clients to be uniquely advantageous. Per DOI theory, the innovation had high relative advantage, relatively low complexity, strong trialability, and compelling observability, key factors of DOI theory that predict successful diffusion. Also, CPs’ contributions were viewed as compatible with staff and client values, norms, and perceived needs, which DOI theory also suggests leads to greater diffusion. Furthermore, we found that staff were able to engage in “reinvention” as needed (e.g., taking a flexible approach to how CPs worked with clients), once they had observed the value of the role and established trust in one another. These characteristics facilitated ongoing implementation, and ultimately informed a consistently expressed desire for sustainability (at the team level), although it was not realized due to constrained resources.
The interview data also suggests that the external facilitation by the research team was able to mitigate the challenges of CPs, or “complexity” in DOI terms, which can undermine diffusion. The research team’s extensive familiarity with CPs and mental health recovery informed site preparations and CP trainings and supervision, which helped avoid common challenges of CP implementation such as dual relationships,6 confidentiality breaches or concerns,6,8 workplace discrimination,7 and stigmatization.8 Further, when challenges did arise, such as a need for workplace skill development, relapse, and some role conflict and confusion, these challenges were generally resolved, respectively, by helping the teams to provide skill building, allowing CPs to return when better, and orienting CPs’ work to their strengths and needs (e.g., assigning a set of clients to a CP who needed structure). This study was consistent with Moll and colleagues,3 who concluded that integration of CPs into teams with no prior CP experience is a process, not an event, and that roles, responsibilities, and relationships evolve over time.
Implications for Behavioral Health
VHA and other healthcare organizations will be hiring more CPs and attention should be paid to implementation issues. External facilitation, guided by an implementation framework like the Simpson Transfer Model, could be an important component of implementation, as is careful planning and anticipation of commonly experienced challenges. For example, within the VHA, Local Recovery Coordinators are clinicians who are often involved in the hiring and supervising of CPs;20 they are a ready workforce that could become involved in the process of introducing CPs into behavioral health services teams. Increasing evidence of successful implementation could contribute to eventual sustainability of this effective addition to behavioral health services.
Acknowledgments
The research reported here was supported by the Department of Veterans Affairs, Veterans Health Administration, Health Services Research and Development Service (IIR 06-227). Support was also provided by the Mental Illness Research, Education, and Clinical Centers of VISN 4 and VISN 22. At the time of the study, Dr. Hamilton was an investigator with the Implementation Research Institute (IRI), at the George Warren Brown School of Social Work, Washington University, St. Louis, through an award from the National Institute of Mental Health (R25 MH080916-01A2) and VA HSR&D QUERI.
Footnotes
Disclaimers: The views expressed in this article are those of the authors and do not necessarily represent the views of the Department of Veterans Affairs or other affiliated institutions.
Conflict of Interest Statement
The authors declare no conflicts of interest exist.
References
- 1.Gates LB, Mandiberg JM, Akabas SH. Building capacity in social service agencies to employ peer providers. Psychiatric Rehabilitation Journal Autumn. 2010;34(2):145–152. doi: 10.2975/34.2.2010.145.152. [DOI] [PubMed] [Google Scholar]
- 2.van Vugt MD, Kroon H, Delespaul PA, et al. Consumer-providers in assertive community treatment programs: associations with client outcomes. Psychiatric Services. 2012;63(5):477–481. doi: 10.1176/appi.ps.201000549. [DOI] [PubMed] [Google Scholar]
- 3.Moll S, Holmes J, Geronimo J, et al. Work transitions for peer support providers in traditional mental health programs: unique challenges and opportunities. Work. 2009;33(4):449–458. doi: 10.3233/WOR-2009-0893. [DOI] [PubMed] [Google Scholar]
- 4.Repper J, Carter T. A review of the literature on peer support in mental health services. Journal of Mental Health Aug. 2011;20(4):392–411. doi: 10.3109/09638237.2011.583947. [DOI] [PubMed] [Google Scholar]
- 5.Wright-Berryman JL, McGuire AB, Salyers MP. A review of consumer-provided services on assertive community treatment and intensive case management teams: implications for future research and practice. Journal of the American Psychiatric Nurses Association Jan-Feb. 2011;17(1):37–44. doi: 10.1177/1078390310393283. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Carlson LS, Rapp CA, McDiarmid D. Hiring consumer-providers: barriers and alternative solutions. Community Mental Health Journal Jun. 2001;37(3):199–213. doi: 10.1023/a:1017569913118. [DOI] [PubMed] [Google Scholar]
- 7.Fisk D, Rowe M, Brooks R, et al. Integrating consumer staff into a homeless outreach project: critical issues and strategies. Psychiatric Rehabilitation Journal. 2000;23(3):244–252. [Google Scholar]
- 8.Gates LB, Akabas SH. Developing strategies to integrate peer providers into the staff of mental health agencies. Administration and Policy in Mental Health and Mental Health Services Research May. 2007;34(3):293–306. doi: 10.1007/s10488-006-0109-4. [DOI] [PubMed] [Google Scholar]
- 9.Stein LI, Test MA. Alternative to mental hospital treatment. I. Conceptual model, treatment program, and clinical evaluation. Archives of General Psychiatry. 1980 Apr;37(4):392–397. doi: 10.1001/archpsyc.1980.01780170034003. [DOI] [PubMed] [Google Scholar]
- 10.Simpson DD. A conceptual framework for transferring research to practice. Journal of Substance Abuse Treatment Jun. 2002;22(4):171–182. doi: 10.1016/s0740-5472(02)00231-3. [DOI] [PubMed] [Google Scholar]
- 11.Chinman M, Shoai R, Cohen A. Using organizational change strategies to guide peer support technician implementation in the Veterans Administration. Psychiatric Rehabilitation Journal Spring. 2010;33(4):269–277. doi: 10.2975/33.4.2010.269.277. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Rogers E. Diffusion of innovations. New York: The Free Press; 1995. [Google Scholar]
- 13.Chinman M, Lucksted A, Gresen R, et al. Early experiences of employing consumer-providers in the VA. Psychiatric Services Nov. 2008;59(11):1315–1321. doi: 10.1176/ps.2008.59.11.1315. [DOI] [PubMed] [Google Scholar]
- 14.Chinman M, Young AS, Hassell J, et al. Toward the implementation of mental health consumer provider services. Journal of Behavioral Health Services and Research Apr. 2006;33(2):176–195. doi: 10.1007/s11414-006-9009-3. [DOI] [PubMed] [Google Scholar]
- 15.Rosenheck R. Stages in the implementation of innovative clinical programs in complex organizations. The Journal of Nervous and Mental Disease Dec. 2001;189(12):812–821. doi: 10.1097/00005053-200112000-00002. [DOI] [PubMed] [Google Scholar]
- 16.Sabin JE, Daniels N. Managed care: Strengthening the consumer voice in managed care: VII. The Georgia peer specialist program. Psychiatric Services Apr. 2003;54(4):497–498. doi: 10.1176/appi.ps.54.4.497. [DOI] [PubMed] [Google Scholar]
- 17.Hasson-Ohayon I, Roe D, Kravetz S. A randomized controlled trial of the effectiveness of the illness management and recovery program. Psychiatric Services Nov. 2007;58(11):1461–1466. doi: 10.1176/ps.2007.58.11.1461. [DOI] [PubMed] [Google Scholar]
- 18.Stetler CB, Legro MW, Rycroft-Malone J, et al. Role of “external facilitation” in implementation of research findings: a qualitative evaluation of facilitation experiences in the Veterans Health Administration. Implementation Science. 2006;1:23. doi: 10.1186/1748-5908-1-23. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Rycroft-Malone J, Kitson A, Harvey G, et al. Ingredients for change: revisiting a conceptual framework. Quality and Safety in Health Care. 2002;11(2):174–180. doi: 10.1136/qhc.11.2.174. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Chinman M, Salzer M, O’Brien-Mazza D. National survey on implementation of peer specialists in the VA: Implications for training and facilitation. Psychiatric Rehabilitation Journal. doi: 10.1037/h0094582. in press. [DOI] [PMC free article] [PubMed] [Google Scholar]