Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2020 Sep 1.
Published in final edited form as: Psychiatr Rehabil J. 2019 Jun 24;42(3):323–328. doi: 10.1037/prj0000369

A Site Visit Protocol for Assessing Recovery Promotion at the Program Level: An Example from the Veterans Health Administration

Nancy J Wewiorski 1, Jay A Gorman 1,7, Marsha Langer Ellison 1,2, Marcia G Hunt 3,4, Leigh Evans 5,6, Martin P Charns 5,6
PMCID: PMC6741775  NIHMSID: NIHMS1037134  PMID: 31233322

Abstract

Objective:

A site visit protocol was developed to assess recovery-promotion in the organizational climate and culture of programs for veterans with serious mental illnesses.

Methods:

The protocol was pilot-tested in four programs: two that had scored high on the pilot version of a staff survey measure of program-level recovery-promotion and two that had scored low. Two-person teams conducted onsite visits and assigned global and organizational domain ratings. Interrater agreement was assessed by examining adjacent agreement and computing weighted kappa.

Results:

The on-site protocol had good interrater agreement and discriminated between sites that scored high and low on the staff survey.

Conclusions and Implications for Practice:

This site visit protocol and procedure shows promise for evaluating recovery-promotion in milieu-based programs. After further refinement of this tool, adaptations could be developed for accreditation protocols or for program self-assessment and quality improvement efforts.

Keywords: Recovery, program evaluation, protocol, veterans, organizational assessment, psychosocial rehabilitation

Impact and Implications

Site visits were conducted to evaluate the extent to which four programs had structures and cultures that promote mental health recovery. We developed an assessment tool that was able to differentiate between programs that were most likely and least likely to promote recovery. After further refinement, this assessment tool could be used in program accreditation and licensing reviews and in-house program quality improvement initiatives.

Recovery is widely endorsed as an organizing principle for mental health service systems (Ellison, Belanger, Niles, Evans & Bauer, 2018), including the Veterans Health Administration (VHA) (Greenberg & Rosenheck, 2009). There have been efforts to operationalize recovery-oriented service systems (Dumont, Ridgway, Onken, Dornan, & Ralph, 2006) and recovery-oriented services (Armstrong & Steffen, 2009; Farkas, Gagne, Anthony & Chamberlin, 2005; O’Connell, Tondora, Croog, Evans & Davidson, 2005), and the VHA has mandated that all Psychosocial Rehabilitation and Recovery Centers (PRRCs) be recovery-oriented (Department of Veterans Affairs, 2011). However, without a clearly articulated model that specifies the essential structures, culture and practices for promoting recovery at the program level, it is difficult to determine the extent to which PRRCs are meeting their recovery-orientation mandate.

The “climcult model” explicates the importance of organizational climate and culture on driving practices within organizations (Ehrhart, Schneider & Macey, 2014). This model posits that culture is reflected in organizational policies, practices, procedures, and leadership. These factors affect organizational climate, and organizational climate is further shaped by factors such as staff rewards, education and training, and quality improvement practices. Although these are critical factors to assess in evaluating the extent to which a mental health program has established a fertile context for recovery-promotion, there currently is no measure that includes key recovery principles and uses a “climcult” organizational framework.

To address this, Evans and colleagues (2019) developed a staff self-report survey measure of recovery-promotion in PRRCs. Using a two-dimensional array of key recovery-principles and key organizational domains, they selected and modified items from 15 existing measures of recovery (Armstrong & Steffen, 2009; Bedregal, O’Connell & Davidson, 2006; Burgess, Pirkis, Coombs & Rosen, 2011; Campbell-Orde, Chamberlin, Carpenter & Leff, 2005; Crowe, Deane, Oades, Caputi, & Morland, 2006; Dumont, Ridgway, Onken, Dornan & Ralph., 2006; Ellis & King, 2003; Higgins, 2008; Mancini, 2006; O’Connell, Tondora, Croog, Evans & Davidson, 2005; Oades, Crowe & Deane, 2007; Ragins, 2010; Ridgway & Press, 2004; Russinova, Rogers, Ellison & Lyass, 2011; Williams, Leamy, Bird, LeBoutillier, Norton, Pesola & Slade 2015) and developed new items to capture recovery principles and organizational domains that were underrepresented in existing measures. Items then were modified and eliminated based on an item review by three recovery experts. The draft survey then was pilot-tested with 6 PRRC staff and further refined based on their feedback about length, organization, flow, and clarity. The final 35-item measure was administered via internet to PRRC staff nationwide in 2017. It was designed to assess recovery-promotion in seven organizational domains (staff expectations; staff values; program leadership; staff training; staff rewards; program policies; and quality improvement) and to incorporate 13 key recovery principles (individualized/person-centered; empowerment; hope; self-direction; relational; non-linear/many pathways; strengths-based; respect; responsibility; peer support; holistic; culture-informed; and trauma-informed) (Ellison et al., 2018). Results are forthcoming in a separate manuscript (Evans, Wewiorski, Hunt, et al, 2019).

On-site reviews are an evaluation methodology commonly used to assess health care program quality. One widely used site visit protocol that assesses recovery-orientation is the Individual Placement and Support (IPS) Fidelity site visit procedure (Bond, Becker, Drake, & Vogler, 1997; Bond, Peterson, Becker, & Drake, 2012), a protocol designed specifically to evaluate supported employment programs. Other similar methods and procedures have been developed for programs such as Assertive Community Treatment (McGrew, Bond, Dietzen & Salyers, 1994; McHugo, Drake, Teague, & Xie, 1999) and Clubhouses (Lucca, 2000). Studies of program evaluation generally conclude that participants have better outcomes in programs that have higher fidelity with proven approaches (Blakely, et al., 1987; Drake, et al., 2001; Mowbray, Holter, Teague, and Bybee, 2003). This study used the IPS-Supported Employment (IPS-SE) procedure as a model for onsite observation to assess the extent to which recovery-promotion was embodied in program-level practices, structures, climate, and culture of PRRCs and to assess the discriminant validity of the Evans measure by comparing survey results with site reviewer ratings in situ.

The VHA Handbook (Department of Veterans Affairs, 2011) gives guidance on procedures and expectations for PRRCs (e.g., staffing, recovery goals, peer support) and CARF (CARF International, 2018) assesses program alignment with VHA policy. By comparison, our protocol focuses on implementation of recovery principles within organizational domains at a depth that illuminates the operationalization of recovery-promotion and it provides an opportunity to gain a nuanced understanding of the recovery context offered by PRRCs. That is, the site visits used the “climcult” lens to more fully understand how recovery is or is not actually promoted at the program level.

Method

Guided by the “climcult” model, the organizational and recovery domains in the Evans PRRC survey measure and the IPS-SE fidelity assessment procedure, the first three authors, all with expertise in mental health recovery, developed and implemented a site visit protocol that was approved by the Bedford VA Institutional Review Board. The protocol included observing the program in operation, interviewing staff and participants, and reviewing documents. Pairs blind to the survey results were assigned to conduct 1.5 day on-site visits to the two PRRCs with the highest average scores (4.5 and 4.6) on the Evans internet survey and the two PRRCs with the lowest average scores (3.1 and 3.4). (Overall survey scores ranged from 3.1 to 4.6, median = 3.9, interquartile range 3.5–4.2).

Site surveyors used a 29-item data collection tool (see Table 1) for recording observations and interview data and for rating the program on 9 organizational domains: 6 domains from the Evans survey (expectations, values, leadership, education/training, staff rewards, quality improvement) and 3 additional domains (structure, coordination, and facility). Semi-structured interview guides with open-ended questions were used to query staff and participants about how the program operated. An observation guide was used to record observations of the program environment, its climate and culture, the interactions between and among staff and participants, and how groups were conducted. After each interview and observation period, each surveyor independently assigned a recovery-promotion rating to each organizational domain about which data had been collected. Ratings were in increments of 0.5 on a scale that ranged from 1 (low) to 5 (high) with behaviorally-defined anchors at the endpoints of each scale.

Table 1.

Examples of Rating Scale Items for Scoring PRRC Programs on Recovery-Promotion.

Domain Example Item Response Anchors
Expectations Participants help/support each other in their recovery journeys 1: No readily observable peer support, no PRRC participants have leadership roles, no PRRC participants have responsibilities in which they work side-by-side with staff
5: Participants or peer staff are in operations meetings, participants work side-by-side with staff during most activities, routine peer support is provided within the PRRC (e.g., Peer staff member is employed), PRRC Veterans assume formal and informal leadership roles
Values How staff handle participant decisions that staff have concerns about 1: Staff ignore or disregard participant’s choice and make unilateral decisions, access to care may change if the participant makes a decision staff oppose (e.g., cannot access some services)
5: Formal process to utilize shared decision-making techniques, staff member empathizes with the participant, acknowledges the participant’s power to choose, and engages in discussion about how this choice relates to their overall goals in a collaborative manner (e.g., open-ended questions and other motivational interviewing techniques), staff indicate they respect the decision, this is conveyed in a non-judgmental manner, no change in care after this decision (e.g., can access all services)
Leadership How program-related decisions are made 1: Changes occur without input from participants, chanses occur prior to participant knowledge
5: Participants have formalized positions/roles (e.g., Veteran voted leadership positions) to direct/advise PRRC programming decisions and there is a structured process as well as open discussion about changes prior to implementation
Staff Education and Training Staff receive training that promotes recovery principles 1: Unclear if staff understands “recovery-oriented principles,” opportunities for staff to provide trainings are not readily available, there are no formal requirements, and little informal encouragement (e.g. email only) to offer in-service training or external trainings to other VA programs or community partners related to recovery-promoting principles
5 : Staff have clear understanding of “recovery-oriented principles,” opportunities to provide training are readily available/accessible and there are annual formal requirements to provide training related to recovery-promoting principles to external VA programs on an annual basis (e.g., CEU presentation), informational discussions with key VA programs (annually), in-service trainings (≥ annually), as well as several external community partner informational discussions on the phone or in-person (≥ quarterly)
Rewards Staff rewards/recognition for championing recovery-promotion 1: No recognition related to championing recovery-oriented principles
5: Routine formal and informal mechanisms to earn recognition for championing recovery-promoting principles including public praise and a clear understanding of how to attain such recognition
Quality Improvement Program participant involvement in quality improvement processes 1: Vague and informal process, no PRRC participant/peer advocate, or specific staff member to report to, no meetings for the purpose of soliciting feedback except for standard program evaluation forms (e.g. NEPEC mandates), little structure to proactively follow-up with the participant on the status of the inquiry
5: Formalized and well known process of reporting, specific meeting to elicit feedback (> quarterly), PRRC participants/peer advocates have a formal role, multiple methods to pursue complaints, suggestions, and requests (e.g., suggestion box, staff, participants), ability to report to multiple staff members, PRRC participants can be involved to advocate on behalf of fellow participants, a formal structure to follow-up on the status of the inquiries
Structure How new program participants are oriented to the PRRC 1: No formal process, tour by staff only, vague explanation about rules, expectations, and opportunities
5: Formalized process in which PRRC participants take a greater or equal role as staff in the orientation process (e.g., battle buddy or several designated PRRC participants share the role) , rules, expectations, and opportunities are clearly explained and given in more than one communication modality (e.g. verbally, written text, video)
Coordination How the program facilitates engagement with natural supports in the community 1: Few natural supports are engaged in the community (e.g., participants do not go to the community, but external partners may engage in the PRRC (< quarterly)
5: Natural supports are engaged in the community in order to meet individualized participant interests, a primary point of contact is readily available to multiple external organizations depending on a participant’s interest
Facility Location and public presentation of the program 1: PRRC is located at the medical center in close proximity to inpatient services. Doors are locked and require staff to open.
5: PRRCs are located in non-institutional community setting with readily accessible public transportation and nearby businesses; Clients have easy access in/out with no excessive security to enter
Open in a new tab

After the site visit, each surveyor compiled all their ratings from interviews and observations and used these data to independently assign one recovery-promotion rating for each organizational domain and a global recovery-promotion rating for the program overall. The two raters then compared their ratings and discussed any differences until they reached a consensus for each domain and for the global rating.

To determine the level of consistency (i.e. interrater agreement) between raters, we assessed adjacent agreement, a method that examines the frequency at which two independent ratings are within one point of each other (Devcich et al., 2016; Jonsson & Svingby (2007). We further examined interrater reliability using weighted kappa (Cohen, 1968), an indicator of the reliability between individual surveyor ratings that accounts for chance agreement between raters and that values adjacent agreements in a progressive manner. We compared the surveyor consensus ratings to Evans’ staff survey results from the corresponding PRRC programs. We plotted the scores by site and examined the difference between the site visit consensus score and the PRRC survey score for each of the 6 domains that were in common for the two assessment methods (i.e. expectations, values, leadership, education/training, rewards, and quality improvement) and for the overall global rating.

Results

Surveyor differences across all the ratings ranged from 0 to 2.0 (see Table 2). The mean difference by organizational domain ranged from 0.25 to 0.75. The mean difference by site ranged from 0.39 to 0.83, and the mean difference in global ratings was 0.13. The median difference for all domain ratings was 0. Among the 36 total domain ratings, 13 (36%) had perfect agreement and 33 (92%) had adjacent agreement. There was 100% adjacent agreement in the global ratings. The weighted kappa statistic measuring all ratings was 62%, indicating substantial agreement between raters (Landis and Koch, 1977).

Table 2.

Interrater Differences in Domain and Global Recovery-Promotion Ratings by Site.

DOMAIN Site A Site B Site C Site D Mean
difference
Expectations 1.0* 0.5* 1.0* 0* 0.63
Values 1.0* 0* 0.5* 0.5* 0.50
Leadership 1.0* 1.0* 0.5* 0.5* 0.75
Education/training 0.5* 0.5* 0* 0* 0.25
Rewards for staff 0* 2.0 0* 1.0* 0.75
Quality improvement 0.5* 0* 0.5* 1.0* 0.50
Structure 0* 2.0 0.5* 0.5* 0.75
Coordination of Care 0* 0* 1.0* 0* 0.25
Location/facility 0* 1.5 1.0* 0* 0.63
Mean difference 0.44 0.83 0.56 0.39 0.56
Global 0.5* 0* 0* 0* 0.13
Open in a new tab
*

Indicates adjacent agreement

Compared to ratings of the site surveyors, staff survey scores tended to be higher, particularly in programs with lower survey scores (see Figure 1). The largest discrepancies between staff survey scores and site surveyor ratings occurred in the programs with the lower survey scores. Although it is not our purpose here to report anecdotal evidence, the surveyors did note striking differences between the high and low sites in the extent to which leadership and program staff embraced recovery and creatively incorporated its principles into the program. For example, staff in the high scoring sites seemed to consistently examine their services, interactions with participants and programmatic decisions through a lens of recovery-promotion.

Figure.1.

Figure.1

Open in a new tab

Comparison of Survey and Site Visit Organizational Domain and Global Ratings of Recovery-Promotion by PRRC Program

Discussion

Consistent with the results of the Evans survey, our site visit protocol using recovery experts as the gold standard and a rating tool with defined response anchors on 29 items covering 9 organizational domains was able to discriminate high from low recovery-promoting PRRC programs. However, the relatively consistent finding of higher domain and global scores on the staff survey compared to the ratings by recovery experts conducting onsite reviews suggest that program evaluations based on a staff survey alone may tend to overrate the extent to which a program is recovery-promoting. Although the rating differences may be due to differences in the construction of the survey and site-visit measures, the somewhat larger discrepancies in the lower scoring sites also could be an indication of social desirability in responses and/or a relative lack of knowledge about recovery by staff working in the lower-scoring programs. In other words, it may be that staff in programs that are low on recovery-promotion “don’t know what they don’t know.” This site visit protocol shows promise as a method for evaluating PRRCs and other milieu-based programs that aim to promote the recovery process of their program participants. The present study provides some preliminary evidence about its usefulness and validity. We found interrater agreement to be quite high. Furthermore, onsite review has the benefit of obtaining more nuanced information about program-level recovery-promotion and also eliminates some of the inherent validity threats in a self-report survey. However, this on-site rating tool needs further development and testing. Some of the rating items and response anchors may require revision. Adaptations of the tool may need to be created for programs that serve specific sub-populations, such as elders or persons with co-occurring disorders. Another improvement may include the addition of persons with lived experience as site surveyors.

After further development and testing of the tool in PRRCs, a future direction may be to test its usefulness for programs outside the VA, such as community-based day treatment programs that are converting to recovery-oriented programs. Future research could also rigorously test the correspondence of site visit ratings with recovery outcomes of service participants. Given the expense of conducting in person site visits, it might be most cost efficient to further develop this tool as a component of accreditation protocols, such as CARF, or as a self-assessment tool for program quality improvement efforts.

Acknowledgments

This study was funded by the VA Health Services Research and Development Service Award PPO 16–135 (P.I. Charns). The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the U.S. Department of Veterans Affairs.

References

  1. Armstrong NP, & Steffen JJ (2009). The recovery promotion fidelity scale: assessing the organizational promotion of recovery. Community Mental Health Journal, 45(3), 163–170. doi: 10.1007/s10597-008-9176-1 [DOI] [PubMed] [Google Scholar]
  2. Bedregal LE, O’Connell M, & Davidson L (2006). The Recovery Knowledge Inventory: assessment of mental health staff knowledge and attitudes about recovery. Psychiatric Rehabilitation Journal, 30(2), 96. [DOI] [PubMed] [Google Scholar]
  3. Blakely CH, Mayer JP, Gottschalk RG, Schmitt N, Davidson WS, Roitman DB, & Emshoff JG (1987). The fidelity-adaptation debate: Implications for the implementation of public sector social programs. American Journal of Community Psychology, 15(3), 253–268. [Google Scholar]
  4. Bond GR, Becker DR, Drake RE, & Vogler KM (1997). A fidelity scale for the individual placement and support model of supported employment. Rehabilitation Counseling Bulletin, 40, 265–284. [Google Scholar]
  5. Bond GR, Peterson AE, Becker DR, & Drake RE (2012). Validation of the revised individual placement and support fidelity scale (IPS-25). Psychiatric Services, 63(8), 758–763. 10.1176/appi.ps.201100476 [DOI] [PubMed] [Google Scholar]
  6. Burgess P, Pirkis J, Coombs T, & Rosen A (2011). Assessing the value of existing recovery measures for routine use in Australian mental health services. Aust N Z Journal of Psychiatry, 45(4), 267–280. doi: 10.3109/00048674.2010.549996 [DOI] [PubMed] [Google Scholar]
  7. CARF International (2018) retrieved from http://www.carf.org/About/WhoWeAre/ on October 18, 2018
  8. Campbell-Orde T, Chamberlin J, Carpenter J, & Leff H (2005). Measuring the promise: A compendium of recovery measures, Volume II . In. Cambridge, MA: Evaluation Center, Human Services Research Institute. [Google Scholar]
  9. Cohen J (1968). Weighted kappa: Nominal scale agreement provision for scaled disagreement or partial credit. Psychological Bulletin, 70(4), 213. [DOI] [PubMed] [Google Scholar]
  10. Crowe TP, Deane FP, Oades LG, Caputi P, & Morland KG (2006). Effectiveness of a collaborative recovery training program in Australia in promoting positive views about recovery. Psychiatric Services, 57(10), 1497–1500. [DOI] [PubMed] [Google Scholar]
  11. Department of Veterans Affairs, Veterans Health Administration (2011). Psychosocial Rehabilitation and Recovery Centers (PRRC) Handbook. Washington, DC: Veterans Health Administration. [Google Scholar]
  12. Devcich DA, Weller J, Mitchell SJ, McLaughlin S, Barker L, Rudolph JW, … Merry AF (2016). A behaviourally anchored rating scale for evaluating the use of the WHO surgical safety checklist: Development and initial evaluation of the WHOBARS. BMJQuality & Safety, 25(10), 778. doi: 10.1136/bmjqs-2015-004448(2016). [DOI] [PubMed] [Google Scholar]
  13. Drake R, Goldman H, Leff H, Lehman A, Dixon L, Mueser K,&Torrey W (2001). Implementing evidence-based practices in routine mental health service settings. Psychiatric Services, 52, 179–182. [DOI] [PubMed] [Google Scholar]
  14. Dumont J, Ridgway P, Onken S, Dornan D, & Ralph R (2006). Mental health recovery: What helps and what hinders? A national research project for the development of recovery facilitating system performance indicators. In: prepared for the National Technical Assistance Center for State Mental Health Planning, National Association of State Mental Health Program Directors, March 2006 [Google Scholar]
  15. Ehrhart MG, Schneider B, & Macey WH (2014). Organizational climate and culture: an introduction to theory, research, and practice. New York, NY: Routledge. [Google Scholar]
  16. Ellis G, & King R (2003). Recovery focused interventions: Perceptions of mental health consumers and their case managers. Australian e-journal for the Advancement of Mental Health, 2(2), 67–76. [Google Scholar]
  17. Ellison ML, Belanger L, Niles B, Evans L, & Bauer M (2018). Explication and Definition of Mental Health Recovery: A Systematic Review. Administration and Policy in Mental Health, 45:91–102. 10.1007/s10488-016-0767-9 [DOI] [PubMed] [Google Scholar]
  18. Evans L, Wewiorski NJ, Hunt MG, Gorman JA, Ni P, Harvey K, Ellison ML, & Charns MP (2019). Development and Validation of an Instrument to Measure Recovery Climate in Mental Health Service Settings for Veterans with Serious Mental Illness. Unpublished Manuscript. [Google Scholar]
  19. Farkas M, Gagne C, Anthony W, & Chamberlin J (2005). Implementing recovery oriented evidence based programs: Identifying the critical dimensions. Community Mental Health Journal, 41(2), 141–158. [DOI] [PubMed] [Google Scholar]
  20. Greenberg GA, & Rosenheck RA (2009). An evaluation of an initiative to improve Veterans Health Administration mental health services: broad impacts of the VHA’s mental health strategic plan. Military Medicine, 174 (12), 1263–1269. 10.7205/MILMED-D-04-3008 [DOI] [PubMed] [Google Scholar]
  21. Higgins A (2008). A recovery approach within the Irish mental health services: A framework for development. Retrieved from: http://hse.openrepository.com/hse/bitstream/10147/75113/1/Framework+for+mental+health+services.pdf
  22. Jonsson A, & Svingby G (2007). The use of scoring rubrics: Reliability, validity and educational consequences. Educational Research Review, 2(2), 130–144. https://doi.org/10.10167i.edurev.2007.05.002 [Google Scholar]
  23. Landis JR, & Koch GG (1977). The measurement of observer agreement for categorical data. Biometrics, 159–174. 10.2307/2529310 [DOI] [PubMed] [Google Scholar]
  24. Lucca AM (2000). A Clubhouse fidelity index: Preliminary reliability and validity results. Mental Health Services Research, 2, 89–94. [DOI] [PubMed] [Google Scholar]
  25. Mancini A (2006). Can recovery orientation inform the implementation of evidence-based practices. Paper presented at the 114th annual convention for the American Psychological Association. [Google Scholar]
  26. McGrew JH, Bond GR, Dietzen L, & Salyers M (1994). Measuring the fidelity of implementation of a mental health program model. Journal of Consulting and Clinical Psychology, 62(4), 670. [DOI] [PubMed] [Google Scholar]
  27. McHugo GJ, Drake RE, Teague GB, & Xie H (1999). Fidelity to assertive community treatment and client outcomes in the New Hampshire dual disorders study. Psychiatric Services, 50(6), 818–824. [DOI] [PubMed] [Google Scholar]
  28. Mowbray CT, Holter MC, Teague GB, & Bybee D (2003). Fidelity criteria: Development, measurement, and validation. American Journal of Evaluation, 24(3), 315–340. [Google Scholar]
  29. Oades LG, Crowe TP, & Deane FP (2007). The Collaborative Recovery Model: moving beyond “us” and “them” in mental health. Faculty of Health and Behavioural Sciences Papers,, 32–38. [Google Scholar]
  30. O’Connell M, Tondora J, Croog G, Evans A, & Davidson L (2005). From rhetoric to routine: assessing perceptions of recovery-oriented practices in a state mental health and addiction system. Psychiatric Rehabilitation Journal, 28(4), 378–386. [DOI] [PubMed] [Google Scholar]
  31. Ragins M (2010). A Recovery Culture Progress Report. Retrieved from author, Los Angeles: [Google Scholar]
  32. Ridgway P, & Press A (2004). Assessing the recovery-orientation of your mental health program: A user’s guide for the Recovery-Enhancing Environment scale (REE). Kansas: University of Kansas. [Google Scholar]
  33. Russinova Z, Rogers ES, Ellison ML, & Lyass A (2011). Recovery-promoting professional competencies: perspectives of mental health consumers, consumer-providers and providers. Psychiatric Rehabilitation Journal, 34(3), 177. [DOI] [PubMed] [Google Scholar]
  34. Williams J, Leamy M, Bird V, Le Boutillier C, Norton S, Pesola F, & Slade M (2015). Development and evaluation of the INSPIRE measure of staff support for personal recovery. Social Psychiatry and Psychiatric Epidemiology, 50(5), 777–786. [DOI] [PubMed] [Google Scholar]

RESOURCES