Development and Evaluation of a Recovery College Fidelity Measure

Abstract

Objective:

Recovery Colleges are widespread, with little empirical research on their key components. This study aimed to characterize key components of Recovery Colleges and to develop and evaluate a developmental checklist and a quantitative fidelity measure.

Methods:

Key components were identified through a systematized literature review, international expert consultation (n = 77), and semistructured interviews with Recovery College managers across England (n = 10). A checklist was developed and refined through semistructured interviews with Recovery College students, trainers, and managers (n = 44) in 3 sites. A fidelity measure was adapted from the checklist and evaluated with Recovery College managers (n = 39, 52%), clinicians providing psychoeducational courses (n = 11), and adult education lecturers (n = 10).

Results:

Twelve components were identified, comprising 7 nonmodifiable components (Valuing Equality, Learning, Tailored to the Student, Coproduction of the Recovery College, Social Connectedness, Community Focus, and Commitment to Recovery) and 5 modifiable components (Available to All, Location, Distinctiveness of Course Content, Strengths Based, and Progressive). The checklist has service user student, peer trainer, and manager versions. The fidelity measure meets scaling assumptions and demonstrates adequate internal consistency (0.72), test-retest reliability (0.60), content validity, and discriminant validity.

Conclusions:

Coproduction and an orientation to adult learning should be the highest priority in developing Recovery Colleges. The creation of the first theory-based empirically evaluated developmental checklist and fidelity measure (both downloadable at researchintorecovery.com/recollect) for Recovery Colleges will help service users understand what Recovery Colleges offer, will inform decision making by clinicians and commissioners about Recovery Colleges, and will enable formal evaluation of their impact on students.

Recovery Colleges are a novel approach to supporting people living with mental health problems through adult education rather than through treatment.¹ They are proposed to be collaborative, strengths based, person centred, inclusive, and community focused.² The concept of ‘recovery education’—supporting recovery in relation to mental health problems through education—was developed in Boston and Phoenix in the 1990s. In the past decade, a model of Recovery Colleges with a greater emphasis on adult learning and coproduction has emerged in the United Kingdom. The first Recovery College opened in 2009, and there are now over 80 operating in the United Kingdom.³ The Recovery College model developed in England has been widely replicated internationally. Sometimes called ‘Discovery Centres’, ‘Empowerment Colleges’, or ‘Recovery Academies’, Recovery Colleges are now open in Australian, Bulgaria, Canada, Hong Kong, Ireland, Italy, Japan, Netherlands, Norway, Poland, and Uganda, among others,⁴ and an international community of practice has been established.⁵ For example, around 5 Recovery Colleges have opened in Canada since 2017,⁶ and a further 15 to 20 are planned to open in the next few years. Recovery Colleges are emerging internationally as a central feature of system transformation towards a recovery orientation.⁷

Available evidence consistently suggests that Recovery Colleges are associated with positive outcomes for students, including increased well-being and achievement of personally valued goals.^8–10 However, most evaluations use uncontrolled and retrospective designs, and despite widespread implementation, only limited evaluative research has been undertaken.⁵ An important knowledge gap is how Recovery Colleges can be distinguished from other forms of treatment and support.

The aim of this study was to characterize the key components of Recovery Colleges and to develop and evaluate a checklist to support the development of Recovery Colleges and a fidelity measure to provide a quantitative fidelity score for use in future evaluations of Recovery Colleges.

Methods

This research was undertaken as part of the Recovery Colleges Characterisation and Testing (RECOLLECT) Study (researchintorecovery.com/recollect). Other elements of the RECOLLECT Study have investigated mechanisms of action and outcomes from Recovery Colleges for students¹¹ and for staff, service, and society,¹² as well as developed a methodology for collaborative data analysis involving people with lived experience.¹³ Ethical committee approval for the RECOLLECT Study was obtained (Nottingham REC 1, 18.1.17, 16/EM/0484). All participants provided written (or verbal when interviewed by phone) informed consent.

Design

In summary, a coding framework identifying key components of Recovery Colleges was iteratively developed through literature review, expert consultation, and semistructured interviews with Recovery College managers. This framework was used as the theory base for the RECOLLECT checklist, a multiperspective assessment of fidelity components to inform local development, and then refined through diverse stakeholder interviews and expert consultation. The checklist was then converted into the RECOLLECT fidelity measure, a single-informant assessment producing a quantitative summary fidelity score. The fidelity measure was evaluated with Recovery College managers, clinicians, and adult education lecturers. This is summarised in Figure 1.

Figure 1.

Study design flow chart.

Setting

The main study sites were Recovery Colleges in Leicester, London, and Sussex in England.

Procedure

A systematized review was conducted.¹⁴ Inclusion criteria were the following: publication relating primarily to Recovery Colleges, proposing fidelity criteria for Recovery Colleges, online publication date 2016 or earlier, available in electronic form, and English language. The exclusion criterion was the following: college prospectus (i.e., course lists for a specific college). Publications were collated from a repository listing published peer-reviewed academic publications (researchintorecovery.com/recoverycolleges); expert consultation with 1) the Implementing Recovery through Organisational Change (ImROC) national transformation program that led to the development of Recovery Colleges in England and internationally (n = 7), 2) international advisory board of 7 experts involved in Recovery Colleges outside England, and 3) the Recovery College International Community of Practice (n = 54); conference abstracts (Refocus on Recovery 2010/2012/2014/2017, ENMESH 2011/2013/2015) with author contact; publications citing included articles using Web of Science; and reference lists of included publications. Fidelity criteria proposals from included papers were synthesized in consultation with ImROC to generate preliminary proposals for a) candidate key components of a Recovery College, b) measurable indicators that could be used to assess each component, c) potential sources of evidence for the indicators, and d) stakeholder perspectives. Refinements following comments by the international advisory board were made to develop a preliminary coding framework.

Semistructured interviews were conducted with Recovery College managers in 10 sites around England between March and May 2017. Sites were chosen to be heterogeneous in terms of geographical spread, commissioning arrangements, longevity and operating model (e.g., one building versus multiple community venues), or differing levels of coproduction. The topic guide comprised open-ended questions such as ‘What are the aims of a Recovery College?’ followed by consultation on the contents of the preliminary coding framework. Interviews were conducted by telephone and immediately transcribed verbatim for thematic analysis. The coding framework was modified based on this analysis and used as the theory basis for an initial checklist, identifying descriptors for ratings of each dimension from 3 perspectives: service user student, defined as a Recovery College student who is using secondary care mental health services now or in the past 2 years; peer trainer, defined as a trainer who has lived experience of mental health challenges and recovery; and Recovery College manager.

The initial checklist was piloted in face-to-face interviews with stakeholders with direct experience of Recovery Colleges in the 3 study sites. Participants completed the initial checklist and were then interviewed about its comprehensiveness, acceptability, and usability. The initial checklist was commented on by 4 expert groups (n = 77 in total): ImROC (n = 7), the international advisory board (n = 7), the Recovery College International Community of Practice (n = 54) comprising international experts in developing or evaluating Recovery Colleges, and a lived experience advisory panel comprising mental health service user Recovery College students, nonstudents, and family members (n = 9).¹³ Refinements produced the finalized coding framework and RECOLLECT Checklist.

The RECOLLECT Checklist was modified by the research team to create a fidelity measure completed by a Recovery College manager to produce a quantitative rating for each component. Between September and November 2018, all Recovery College managers in England (n = 75) were asked to complete this fidelity measure twice 2 weeks apart and to provide feedback on face and content validity, comprehensiveness, acceptability, and usability either by email or though cognitive debriefing¹⁵ in person where feasible. To investigate discriminant validity, the fidelity measure was completed by a) clinicians in the 3 study sites in relation to psychoeducational courses they provided in adult mental health services (i.e., not in Recovery Colleges) and b) adult education college lecturers local to the study sites in relation to their college courses. Refinements were made following feedback and psychometric evaluation to produce the finalized RECOLLECT Fidelity Measure, with minor adjustments made to the RECOLLECT Checklist to ensure consistency.

Analysis

Qualitative data used to develop the coding framework were analyzed using the framework method.¹⁶ Initial proposals for candidate components were used to shape the preliminary coding framework, which was further developed through open coding and an iterative process of individual analysis and joint discussion between 6 researchers with backgrounds in psychotherapy, occupational therapy, clinical psychology, and social anthropology, spanning junior and senior roles, as well as including people with personal and family experience of mental health issues. This allowed the emergence of unanticipated categories rather than restricting the investigation to predetermined concepts or prejudging the significance of concepts.

Psychometric evaluation used correlational and descriptive analyses for data quality (missing data), scale assumptions (legitimacy of summing items, using similarity of item means and variances, magnitude and similarity of corrected item-total correlations), scale-to-sample targeting (score means and standard deviation, floor and ceiling effects), and reliability (Cronbach’s α, test-retest). A rating of ‘type 2’ was arbitrarily assumed to be higher fidelity. Fit, defined as the extent to which items capture the fidelity of Recovery Colleges, was tested by visually inspecting a) the ordering of the response options, b) the ordering of the item thresholds, and c) 2 statistical indicators: item fit residuals (±2.5) and χ². Discriminant validity was evaluated using independent sample tests comparing total and item-level scores on ratings by Recovery Colleges and a) clinicians and b) adult education lecturers.

Results

Key Components

Thirteen publications were included (Online Supplement 1). Primary published sources that informed the coding framework were a briefing paper on Recovery Colleges¹ and a single-site study of Recovery College characteristics.² Other publications were overviews of recovery college components¹⁷ and of emerging communities of practice,^5,18 reviews of key aspects (coproduction,^19–21 outcomes,^8,22 recovery),²³ and preliminary evaluations of impacts on staff²⁴ and services.²⁵

The coding framework after literature review and expert consultation comprised 7 components and 12 measurable indicators (Table 1, column 1).

Table 1.

Summary of the Development of the RECOLLECT Checklist and RECOLLECT Fidelity Measure.

Coding Framework after Literature Review and Expert Consultation	Coding Framework after Manager Interviews	Final Coding Framework Used in RECOLLECT Checklist and RECOLLECT Fidelity Measure
Components Education Coproduction Strengths Based Person Centred Progressive Community Focused Inclusive Measurable indicators Physical base College principles Self-referral Course selection Personal tutors Individual learning plan Not assessment and treatment Not mainstream college Recovery principles Free of charge A safe place Empathic, warm, and welcoming staff	Nonmodifiable components Valuing Equality Learning Individualized Experience Coproduction Community Focus Passion Modifiable components Eligibility Location Course Distinctiveness Strengths Based Progressive Perspectives Service user student Peer trainer Recovery College manager	Nonmodifiable components Versions: service user students, peer trainers, Recovery College manager Scoring: 3-point ordinal scale [Checklist: Red-Amber-Green, Measure: 0, 1, 2] Rating for highest score (student version) shown Valuing Equality—I am treated with respect and my views are valued. Learning—I feel fully involved in my learning and my ideas are valued by everyone. Tailored to the Student—I am provided with lots of support for my personal needs. Coproduction of the Recovery College—All of the courses I attend are delivered by a peer trainer and someone else. I can be regularly involved in shaping how the Recovery College is run. Social Connectedness—The college feels relaxed and I have time to get to know other students during courses and when using the college’s other facilities. Community Focus—Lots of courses are delivered by community organizations and I am aware of how I can be supported to move from the college to being in a community organization for ongoing support or activity. Commitment to Recovery—Staff are passionate and dedicated to recovery. Modifiable components Version: Recovery College manager  Scoring: binary rating [type 1 or type 2]. See Table 2 for anchor points. Available to All Location Distinctiveness of Course Content Strengths Based Progressive

RECOLLECT, Recovery Colleges Characterisation and Testing.

Interviews with 10 Recovery College managers were conducted to refine the coding framework, comprising 6 modifiable and 5 nonmodifiable components (Table 1, column 2). For nonmodifiable components, Education became Learning as most participants voiced an opinion that the word education is reminiscent for students of school and does not capture the adult learning ethos: ‘It’s a very different form of education because it’s an engagement in the ideas…you’re not just learning it, you’re trying it out. You learn by the method of trying’ (#1). Person Centred became Individualized Experience, reflecting the language choices of interviewees. The Valuing Equality and Passion components were added because interviewees made repeated and emphatic references to challenging stigma and discrimination and to the investment of personal, emotional energy:

The balance is totally different and we are more sort of partners. We work together. Rather than staff doing something for patients, it’s more sort of, coaching (#4).

[A basic definition of a Recovery College] misses the passion. The impact. The kind of emotional impact that Recovery Colleges have. I think that unfortunately in this day and age with everything that’s going on in services and particularly within mental health services…I think it’s [passion] less and less likely to be found in other places (#3).

The 5 modifiable components described characteristics, defined in Table 2, in which individual Recovery Colleges operate in 1 of 2 distinct ways. Each modifiable component is independent of the others, so Recovery Colleges could be type 1 on some modifiable components and type 2 on others.

Table 2.

Components (n = 12) of the RECOLLECT Checklist and RECOLLECT Fidelity Measure.

Nonmodifiable Components	Description
1. Valuing Equality	Relationships between all students, peer trainers, nonpeer trainers, and other staff are nondiscriminatory and respectful of diversity. No one is judged or treated differently because of their background or mental health difficulties, and everyone’s contribution is equally valued.
2. Learning	Recovery Colleges follow an adult education approach whereby students and trainers collaborate and learn from each other by sharing experiences, knowledge, and skills. Students have responsibility for their learning and learn through interactive and reflective exercises. Students gain self-awareness, understanding of their difficulties, and practical, relevant self-management skills. Students choose courses that best suit their interests and aspirations.
3. Tailored to the Student	Recovery Colleges do not offer a one-size-fits-all experience. Students’ individual needs are actively enquired about and accommodated during courses (e.g., personalized handouts, translated text, materials adapted for learning difficulties). Their needs outside the course are also accommodated (e.g., buddy service, transport help, individual learning plans).
4. Coproduction of the Recovery College	People with lived experience (peer trainers and students) are brought together with staff and professional/subject experts to design and deliver all aspects of the Recovery College. This includes collaborative decision making about the prospectus, courses, college policies, staff recruitment, advertising, and so on, as well as the codesign and codelivery of all courses by a peer and nonpeer trainer.
5. Social Connectedness	The culture and the physical environment of the college provide students with opportunities to develop informal, meaningful connections with others. The learning space is relaxed (e.g., nonclinical chair layout, access to drinks facilities, shared spaces for socializing). Trainers recognize and cater for students’ social needs (e.g., organizing exercises and breaks for chatting, sharing experiences and developing friendships).
6. Community Focus	Recovery Colleges engage with community organizations (e.g., mental health charities, artistic/sporting groups) and Further Education colleges to coproduce relevant courses. The college provides students with information, handouts, and events that support students’ pathways into valued activities, roles, relationships, and support in the community.
7. Commitment to Recovery	Recovery Colleges are passionate places where staff talk with conviction and enthusiasm about the service and are dedicated to students’ recovery. There is a tangible energy to the college and its activity and an expression of shared values about the recovery principles on which the college is based.
Modifiable components	Type 1 vs type 2
8. Available to All	Only minimal restrictions (e.g., aged 18+) vs limited to specific groups (e.g., mental health service users, staff and family members).
9. Location	In a community location not connected with services vs in a location shared with services.
10. Distinctiveness of Course Content	Any topic can be offered as a course vs only topics not available in mainstream adult education settings are offered.
11. Strengths Based	The focus on strengths (not problems) is implicit vs explicit.
12. Progressive	The focus is on ‘being’ and ‘becoming’ (not on goal setting) vs the focus is on ‘becoming’.

RECOLLECT, Recovery Colleges Characterisation and Testing.

RECOLLECT Checklist

The initial checklist was developed based on the 11 identified components, with different versions for service user students, peer trainers, and Recovery College managers. It was evaluated in interviews with 3 Recovery College managers, 11 peer and nonpeer trainers, and 30 service user students. Interview participants emphasized the importance of Social Connectedness, which was introduced as a new component. The component Passion was rephrased to Commitment to Recovery. Language was made more accessible, and more indicators and examples of evidence were given. The final coding framework comprises 12 components (Table 1, column 3).

The framework was used to finalize the RECOLLECT Checklist (Online Supplement 2) and the RECOLLECT Fidelity Measure.

RECOLLECT Fidelity Measure

Thirty-nine (52%) of the 75 Recovery College managers in England completed the RECOLLECT Fidelity Measure and provided feedback in interview (n = 8) or by telephone/email (n = 31), and 23 (59%) recompleted the Fidelity Measure 2 weeks later. Eleven clinicians completed the measure in relation to their psychoeducational groups in National Health Service (NHS) adult mental health services, and 10 lecturers from local adult (18+) further education colleges completed the measure in relation to their college courses. Descriptions of key components and anchor points were refined, and completion by a group of key informants rather than just the Recovery College manager was allowed. The description of key components (Table 2) and the RECOLLECT Fidelity Measure (Online Supplement 3) was finalized.

Data quality was high, scaling assumptions were met (items had similar mean and scale scores spanning the measurement continuum), no floor/ceiling effects were found, Cronbach’s α (0.72) and test-retest intraclass correlation coefficients (0.60) were acceptable, and κ coefficient (0.48) for items 8 to 12 indicated moderate agreement, providing initial evidence for reliability (Table 3). Item-level intraclass correlation coefficients (range, 0.63 to 0.81) were above the suggested minimum of 0.50.

Table 3.

Data Quality, Scaling Assumptions, Targeting, and Reliability for RECOLLECT Fidelity Measure (n = 39).

Psychometric Property	Total
Data quality
Missing data time 1 (%)	2.4
Missing data time 2 (%)	0.4
Scaling assumptionsa
Items 1-7 mean scores: mean (range)	1.58 (1.33-1.87)
Items 1-7 SD: range	0.41-0.67
Items 8-12 mean scores: mean (range)	0.55 (0.41-0.69)
Items 8-12 SD: range	0.49-0.50
Targeting
Mean score (SD)	13.73 (2.55)
Possible score rangeb	0-19
Observed score range	8-18
Floor/ceiling effectc	0%/0%
Reliability of items 1-7
Cronbach’s α	0.72
Test-retest	0.60
Mean interitem correlation	0.02-0.49
Reliability of items 8-12
κ coefficient	0.48

RECOLLECT, Recovery Colleges Characterisation and Testing.

^a Items 1-7 have an ordinal response scale scored 0 to 3; items 8 to 12 have a dichotomous response scale, scored as 0 or 1.

^b Higher scores indicate higher fidelity.

^c Floor effect = % receiving a score of 0; ceiling effect = % receiving a score of 19.

In relation to construct validity, the fit of items was consistent with the item-person threshold map (not shown). In Table 4, items are listed in terms of easiest (indicating lower fidelity if not endorsed) to most difficult (indicating higher fidelity if endorsed). This item hierarchy (i.e., the construct validity) can be interpreted to be an ordered list of fidelity items, with Coproduction of the Recovery College (item 4) and Learning (item 2) emerging as the easiest items to endorse, and Available to All (item 8), Strengths Based (item 11), and Distinctiveness of Course Content (item 10) as most difficult. The Location component (item 9) showed evidence for borderline misfit. Removing this borderline redundant item did not improve internal consistency, but given the theoretical rationale for including the item, it was retained for future testing. The hierarchy of item difficulties provides evidence to support the intentions of the measure and informs how the total score can be interpreted in a clinically meaningful way (i.e., clinical utility). The ordering of the items also informs implementation approaches.

Table 4.

Measures of Fit and Location (SE) of RECOLLECT Fidelity Measure Items (n = 39).

Component	Location	SE	Fit Residual	χ2 df (32,2)	P Value
4. Coproduction of the Recovery College	–3.70	0.37	–0.26	3.44	0.179
2. Learning	–3.53	0.36	0.03	6.84	0.033
1. Valuing Equality	–0.58	0.30	–0.22	4.04	0.132
7. Commitment to Recovery	–0.55	0.30	–2.15	9.42	0.009
5. Social Connectedness	–0.05	0.33	0.70	3.74	0.154
3. Tailored to the Student	–0.04	0.31	–0.01	5.92	0.053
6. Community Focus	0.63	0.26	–0.03	7.50	0.024
12. Progressive	1.21	0.39	1.54	7.00	0.030
9. Location	1.27	0.39	3.20	18.59	<0.001
8. Available to All	1.41	0.38	1.99	5.77	0.059
11. Strengths Based	1.57	0.38	1.42	4.81	0.090
10. Distinctiveness of Course Content	2.37	0.38	0.92	2.91	0.233

RECOLLECT, Recovery Colleges Characterisation and Testing. Bold signifies p < 0.001.

Total scores for Recovery College managers for their Recovery College (mean [SD], 13.73 [2.55]) were significantly different compared with clinicians rating their psychoeducational groups (mean [SD], 7.36 [2.41]) (t = 7.58, P < 0.01). All items showed strong evidence for discriminating except component 11 (Strengths Based). There was no significant difference between scores for Recovery College managers rating their Recovery College and adult education lecturers rating their further education courses (t = 0.710, P = 0.480), but at the component level, significant differences were found for Coproduction of the Recovery College (t = 3.10, P = 0.003) and Progressive (t = 2.470, P = 0.016). The differences found in both comparisons were due to Recovery College managers rating higher fidelity than the comparator group.

Discussion

This mixed-methods study identified 7 nonmodifiable and 5 modifiable components of Recovery Colleges. A new checklist to support Recovery College development and a new fidelity measure supporting Recovery College evaluation were developed and evaluated. The fidelity measure has good internal consistency, adequate test-retest reliability, and good content validity, and it can differentiate Recovery Colleges from clinician-run psychosocial groups and adult education courses. Other RECOLLECT studies have characterized the mechanisms of action and outcomes for mental health service user students¹¹ and staff¹² attending Recovery Colleges. Together, these provide a theory of change for Recovery Colleges, characterizing what they do and their impact.

Rasch analysis found that Coproduction of the Recovery College and Learning were the most likely components to be endorsed, so if they are not high, then other fidelity components are less likely to be endorsed. Therefore, use of coproduction and adult learning approaches should be the initial focus in developing a new Recovery College, and once these are achieved, other components should be prioritized as per the ordered list in Table 4.

A key paper identified in the review was a single-site study identifying 7 critical dimensions (Educational, Collaborative, Strengths Based, Person Centred, Progressive, Community Focused, Inclusive),² which have been informally published as an unvalidated fidelity measure.²⁶ Extensions in the current study were collection of data from over half of the 75 Recovery Colleges in England, allowing identification of modifiable components, more detailed evaluation of proposed concepts and language across a wide range of stakeholders, and the development, preliminary psychometric evaluation, and publication of a checklist and measure. Components identified in our study also map onto the findings from a recent systematic review.⁹ Based on analysis of 77 included publications, the authors highlighted the central importance of an educational approach and of codesign, aligning with our findings that coproduction and learning are the foundational components of a Recovery College.

Coproduction has been identified as a core value for psychiatrists,²⁷ reflecting the increasing focus in general on coproduction in health care.²⁸ A reported strength of Recovery Colleges is that they provide an alternative space in which a coproductive culture can more easily emerge than in traditional mental health services. Contrasts include use of pedagogical approaches such as transformative learning theory²⁹ as the underpinning model, within which active engagement is assumed; the use of more socially valued labels (e.g., ‘student’ not ‘patient’ or ‘service user’); a reduced focus on risk management; and a stronger emphasis on the ethical values of autonomy and justice rather than on beneficence and nonmaleficence.³⁰ One way in which coproduction is enacted in Recovery Colleges is in the planning and delivery of training, typically involving a peer trainer bringing lived experience and a nonpeer trainer bringing professional expertise. Hope is central to recovery,³¹ and attending courses codelivered with peer trainers both gives students contact with ‘credible role models of recovery’³² in the peer trainer and exposes them to potentially more partnership-based clinician-service user relationships, both of which increase hope.³³ The UK model of Recovery Colleges retains a focus on involvement from health professionals, so it cannot be described as a peer-led³⁴ approach. Other models emerging internationally have more peer leadership and less professional involvement¹⁸; these models raise different questions not addressed in the current study, such as whether professionals are sufficiently involved in coproduction and how the Recovery College can affect mental health system culture.

We found a consensus that a focus on learning is central to Recovery Colleges. The success of Recovery Colleges may be attributable to this focus—students like to learn what Recovery Colleges offer, they like the way that courses are delivered, and learning improves well-being.^35,36 However, few participants talked about theoretical aspects of education, such as situated learning³⁷ and collaborative construction of knowledge.³⁸ A study interviewing 10 psychiatrists about their views on Recovery Colleges found that they viewed the approach positively as a form of service user involvement, whilst expressing concerns about their approach to risk management and safeguarding issues, and whether they may encourage medication nonadherence.³⁹ Future research should clarify the extent to which these concerns relates to the specific Recovery College, perhaps assessed using the RECOLLECT Fidelity Measure, or more conceptual concerns relating to the development of nonmedical discourses about, for example, the role of medication and mental health services in recovery.

Other nonmodifiable components of Recovery Colleges were Valuing Equality and Commitment to Recovery. These were often expressed as subcultural values held in the Recovery College and typically described as being in contrast to the wider mental health system, reflecting wider debates about recovery and medicine.⁴⁰ Stigma against people using mental health systems is a known problem,⁴¹ and Recovery Colleges seem to offer a space of acceptance. Both staff and student respondents identified benefits arising from a reduced emphasis on hierarchies of power, less of a ‘them and us’ distinction, and the creation of a space in which passion about recovery was possible, reflecting an organizational commitment to recovery.⁴² Similarly, supporting Social Connectedness was a particular focus in Recovery Colleges, reflecting the established importance of connectedness^23,43,44 and social capital^33,45,46 for recovery.

None of the studies included in our review as proposing Recovery College fidelity criteria^1,2,4,9 identified modifiable components. Recovery Colleges can be understood as a complex intervention, defined as one in which flexibility and tailoring of the intervention are permitted.⁴⁷ The most recent overview of Recovery Colleges in England identified the need for more robust research but cautioned that ‘it is important that this does not ossify what is a continually evolving creation’ (p. 32).⁴ A balance needs to be struck between defining the necessary features of a Recovery College whilst encouraging ongoing innovation, and our identification and defining of nonmodifiable components (without which the service is not a Recovery College) and modifiable components (for which local tailoring is possible) provides an approach to striking this balance.

Limitations can be identified. The systematized search strategy may have missed key publications, although a systematic review published after the study⁹ did not identify any relevant papers not included in our review. The psychometric evaluation is based on a small sample size, although it includes responses from over half of all Recovery Colleges in England, so the psychometric characteristics of the RECOLLECT Fidelity Measure for other Recovery Colleges in England is unknown. Finally, no staff students were involved in the interviews, and the RECOLLECT checklist has versions only for service user students, peer trainers, and Recovery College managers. Future work might develop and evaluate versions of the checklist for staff students and nonpeer trainers.

The preliminary psychometric evaluation of the fidelity measure indicated further work may be needed to strengthen the test-retest reliability (e.g., by improving the anchor points), to validate the item hierarchy (e.g., using qualitative methods), and to investigate whether the Location component can be adequately rated. Once finalized, future work will need to investigate the relationship between the 12 components and outcomes, to validate the RECOLLECT measure, and to establish whether any of the nonmodifiable elements can be modified and vice versa. Given the international spread of Recovery Colleges, it will also be important to establish cross-cultural validity of this UK-developed measure. Just as other recovery interventions require cross-cultural modification, such as peer support work,⁴⁸ the conceptual equivalence of measured concepts such as ‘equality’, ‘community’, and ‘coproduction’ will need to be established in other settings. Some items are based on assumptions; for example, the Location item is premised on the assumption that a health and social care system exists, which may not be the case in low-income settings.

The item hierarchy provides an ordering to inform interventions to improve Recovery College fidelity, and the initial focus should be on establishing coproduction and an adult learning environment before addressing other components.

Conclusions

There is a strong business case for Recovery Colleges as part of a broader reorientation of mental health systems towards recovery,⁴⁹ yet no trials have been published.⁹ This study provides a basis for fidelity evaluation in a randomized controlled trial evaluation of Recovery Colleges.