Interventions for obtaining and maintaining employment in adults with severe mental illness, a network meta‐analysis

Yvonne B Suijkerbuijk; Frederieke G Schaafsma; Joost C van Mechelen; Anneli Ojajärvi; Marc Corbière; Johannes R Anema

doi:10.1002/14651858.CD011867.pub2

. 2017 Sep 12;2017(9):CD011867. doi: 10.1002/14651858.CD011867.pub2

Interventions for obtaining and maintaining employment in adults with severe mental illness, a network meta‐analysis

Yvonne B Suijkerbuijk ^1,², Frederieke G Schaafsma ^1,^2,^✉, Joost C van Mechelen ¹, Anneli Ojajärvi ³, Marc Corbière ^4,⁵, Johannes R Anema ^1,²

Editor: Cochrane Work Group

PMCID: PMC6483771 PMID: 28898402

Abstract

Background

People with severe mental illness show high rates of unemployment and work disability, however, they often have a desire to participate in employment. People with severe mental illness used to be placed in sheltered employment or were enrolled in prevocational training to facilitate transition to a competitive job. Now, there are also interventions focusing on rapid search for a competitive job, with ongoing support to keep the job, known as supported employment. Recently, there has been a growing interest in combining supported employment with other prevocational or psychiatric interventions.

Objectives

To assess the comparative effectiveness of various types of vocational rehabilitation interventions and to rank these interventions according to their effectiveness to facilitate competitive employment in adults with severe mental illness.

Search methods

In November 2016 we searched CENTRAL, MEDLINE, Embase, PsychINFO, and CINAHL, and reference lists of articles for randomised controlled trials and systematic reviews. We identified systematic reviews from which to extract randomised controlled trials.

Selection criteria

We included randomised controlled trials and cluster‐randomised controlled trials evaluating the effect of interventions on obtaining competitive employment for adults with severe mental illness. We included trials with competitive employment outcomes. The main intervention groups were prevocational training programmes, transitional employment interventions, supported employment, supported employment augmented with other specific interventions, and psychiatric care only.

Data collection and analysis

Two authors independently identified trials, performed data extraction, including adverse events, and assessed trial quality. We performed direct meta‐analyses and a network meta‐analysis including measurements of the surface under the cumulative ranking curve (SUCRA). We assessed the quality of the evidence for outcomes within the network meta‐analysis according to GRADE.

Main results

We included 48 randomised controlled trials involving 8743 participants. Of these, 30 studied supported employment, 13 augmented supported employment, 17 prevocational training, and 6 transitional employment. Psychiatric care only was the control condition in 13 studies.

Direct comparison meta‐analysis of obtaining competitive employment

We could include 18 trials with short‐term follow‐up in a direct meta‐analysis (N = 2291) of the following comparisons. Supported employment was more effective than prevocational training (RR 2.52, 95% CI 1.21 to 5.24) and transitional employment (RR 3.49, 95% CI 1.77 to 6.89) and prevocational training was more effective than psychiatric care only (RR 8.96, 95% CI 1.77 to 45.51) in obtaining competitive employment.

For the long‐term follow‐up direct meta‐analysis, we could include 22 trials (N = 5233). Augmented supported employment (RR 4.32, 95% CI 1.49 to 12.48), supported employment (RR 1.51, 95% CI 1.36 to 1.68) and prevocational training (RR 2.19, 95% CI 1.07 to 4.46) were more effective than psychiatric care only. Augmented supported employment was more effective than supported employment (RR 1.94, 95% CI 1.03 to 3.65), transitional employment (RR 2.45, 95% CI 1.69 to 3.55) and prevocational training (RR 5.42, 95% CI 1.08 to 27.11). Supported employment was more effective than transitional employment (RR 3.28, 95% CI 2.13 to 5.04) and prevocational training (RR 2.31, 95% CI 1.85 to 2.89).

Network meta‐analysis of obtaining competitive employment

We could include 22 trials with long‐term follow‐up in a network meta‐analysis.

Augmented supported employment was the most effective intervention versus psychiatric care only in obtaining competitive employment (RR 3.81, 95% CI 1.99 to 7.31, SUCRA 98.5, moderate‐quality evidence), followed by supported employment (RR 2.72 95% CI 1.55 to 4.76; SUCRA 76.5, low‐quality evidence).

Prevocational training (RR 1.26, 95% CI 0.73 to 2.19; SUCRA 40.3, very low‐quality evidence) and transitional employment were not considerably different from psychiatric care only (RR 1.00,95% CI 0.51 to 1.96; SUCRA 17.2, low‐quality evidence) in achieving competitive employment, but prevocational training stood out in the SUCRA value and rank.

Augmented supported employment was slightly better than supported employment, but not significantly (RR 1.40, 95% CI 0.92 to 2.14). The SUCRA value and mean rank were higher for augmented supported employment.

The results of the network meta‐analysis of the intervention subgroups favoured augmented supported employment interventions, but also cognitive training. However, supported employment augmented with symptom‐related skills training showed the best results (RR compared to psychiatric care only 3.61 with 95% CI 1.03 to 12.63, SUCRA 80.3).

We graded the quality of the evidence of the network ranking as very low because of potential risk of bias in the included studies, inconsistency and publication bias.

Direct meta‐analysis of maintaining competitive employment

Based on the direct meta‐analysis of the short‐term follow‐up of maintaining employment, supported employment was more effective than: psychiatric care only, transitional employment, prevocational training, and augmented supported employment.

In the long‐term follow‐up direct meta‐analysis, augmented supported employment was more effective than prevocational training (MD 22.79 weeks, 95% CI 15.96 to 29.62) and supported employment (MD 10.09, 95% CI 0.32 to 19.85) in maintaining competitive employment. Participants receiving supported employment worked more weeks than those receiving transitional employment (MD 17.36, 95% CI 11.53 to 23.18) or prevocational training (MD 11.56, 95% CI 5.99 to 17.13).

We did not find differences between interventions in the risk of dropouts or hospital admissions.

Authors' conclusions

Supported employment and augmented supported employment were the most effective interventions for people with severe mental illness in terms of obtaining and maintaining employment, based on both the direct comparison analysis and the network meta‐analysis, without increasing the risk of adverse events. These results are based on moderate‐ to low‐quality evidence, meaning that future studies with lower risk of bias could change these results. Augmented supported employment may be slightly more effective compared to supported employment alone. However, this difference was small, based on the direct comparison analysis, and further decreased with the network meta‐analysis meaning that this difference should be interpreted cautiously. More studies on maintaining competitive employment are needed to get a better understanding of whether the costs and efforts are worthwhile in the long term for both the individual and society.

Plain language summary

Helping adults with severe mental illness get a job and to keep it, a network meta‐analysis

What is the aim of this review?

The aim of this review was to find out if it is possible to help adults with severe mental illness get a job and to keep it.

People with severe mental illness, such as schizophrenia or bipolar disorder, are more often unemployed. However, these people still often have a desire to work. There are many ways to try and help them obtain a competitive job. People with severe mental illness used to be placed in sheltered employment or they were enrolled in prevocational training, before searching for competitive work. Now there are also interventions focusing directly on finding a job quickly, with ongoing support to keep the job. This is known as supported employment. Recently, there has been a growing interest in combining supported employment with other prevocational or psychiatric interventions.

Key messages

Supported employment and augmented supported employment are more effective than the other interventions in obtaining and maintaining competitive employment for people with severe mental illness without increasing the risk for hospital admissions. The difference in effectiveness between supported employment and augmented supported employment is small. Future research should evaluate the cost‐effectiveness of augmented supported employment compared to supported employment only.

What was studied in the review?

We included 48 randomised controlled trials involving 8743 participants. The interventions included prevocational training, transitional employment, such as sheltered jobs, supported employment, supported employment augmented with other specific interventions or psychiatric care only. We used the data from these studies about the number of participants who obtained a competitive job and the number of weeks they worked. Through a direct comparison meta‐analysis and a network meta‐analysis we assessed the difference in effectiveness between all interventions, and ranked these accordingly.

What are the results of the review?

Supported employment and augmented supported employment are more effective than prevocational training, transitional employment or psychiatric care only in obtaining employment in both types of meta‐analysis. In the direct comparison meta‐analysis prevocational training was also more effective than psychiatric care only. Augmented supported employment shows slightly better results than supported employment alone, again in both types of meta‐analysis. However, this result was less clear in the network meta‐analysis. In the subgroup analysis supported employment with symptom‐related skills training showed the best results. The results are based on moderate‐ to very low‐quality evidence, meaning that the results of future studies could change our conclusions. Augmented supported employment is more effective than prevocational training and supported employment in maintaining competitive employment in the direct comparison meta‐analysis. The results favour supported employment compared to transitional employment in maintaining competitive employment.

Overall, we did not find any differences between interventions in the risk of participants dropping out or hospital admissions.

How up to date is this review?

We searched for studies that had been published up to 11 November 2016.

Summary of findings

Summary of findings for the main comparison. Summary of findings of network meta‐analysis.

Patient or population: adults with severe mental illness Settings: (community) psychiatric care/mental health services Interventions/comparisons: interventions for obtaining competitive employment: augmented supported employment, supported employment. pre‐vocational training, transitional employment, psychiatric care only
Comparison	Illustrative comparative risks^a (95% CI)		Relative effect  (95% CI)	SUCRA	No of participants  (studies with direct evidence) ^b	Quality of the evidence  (GRADE)^c
Comparison	Assumed likelihood with control intervention	Corresponding likelihood with intervention	Relative effect  (95% CI)	SUCRA	No of participants  (studies with direct evidence) ^b	Quality of the evidence  (GRADE)^c
Outcome: Number of participants who obtained competitive employment (follow up > 1 year)
Augmented supported employment vs. psychiatric care only	187 per 1000 (18.7%)	712 per 1000 (372 to 1366)	RR 3.81 (1.99 to 7.31)	98.5%	256 (1 study)	⊕⊕⊕⊝  moderate¹
Supported employment vs. psychiatric care only	187 per 1000 (18.7%)	509 per 1000 (290 to 890)	RR 2.72 (1.55 to 4.76)	76.5%	2238 (1 study)	⊕⊕⊝⊝  low²
Pre‐vocational training vs. psychiatric care only	187 per 1000 (18.7%)	236 per 1000 (136 to 410)	RR 1.26 (0.73 to 2.19)	40.3%	161 (2 studies)	⊕⊝⊝⊝  very low³
Transitional employment vs. psychiatric care only	187 per 1000 (18.7%)	187 per 1000 (95 to 367)	RR 1.00 (0.51 to 1.96)	17.2%	0	⊕⊕⊝⊝  low ⁴
Augmented supported employment vs. transitional employment	223 per 1000 (22.3%)	845 per 1000 (522 to 1369)	RR 3.79 (2.34 to 6.14)		212 (2 studies)	⊕⊕⊝⊝  low⁵
Supported employment vs. transitional employment	223 per 1000 (22.3%)	604 per 1000 (401 to905)	RR 2.71 (1.80 to 4.06)		87 (4 studies)	⊕⊕⊕⊝  moderate⁶
Pre‐vocational training vs. transitional employment	223 per 1000 (22.3%)	281 per 1000 172 to 457)	RR 1.26 (0.77 to 2.05)		0	⊕⊕⊝⊝  low⁷
Augmented supported employment vs. pre‐vocational training	263 per 1000 (26.3%)	794 per 1000 (494 to 1280)	RR 3.02 (1.88 to 4.87)		193 (2 studies)	⊕⊕⊝⊝  low⁸
Supported employment vs prevocational training	263 per 1000 (26.3%)	568 per 1000 (419 to 771)	RR 2.16 (1.59 to 2.93)		1569 (9 studies)	⊕⊝⊝⊝  very low⁹
Augmented supported employment vs supported employment only	457 per 1000 (45.7%)	640 per 1000 420 to 978)	RR 1.40 (0.92 to 2.14)		205 (3 studies)	⊕⊕⊝⊝  low¹⁰
GRADE Working Group grades of evidence  High quality: we are very confident that the true effect lies close to that of the estimate of the effect.  Moderate quality: we are moderately confident in the effect estimate: the true effect is likely to be close to the estimate of the effect, but there is a possibility that it is substantially different.  Low quality: our confidence in the effect estimate is limited; the true effect may be substantially different from the estimate of the effect.  Very low quality: we have very little confidence in the effect estimate: the true effect is likely to be substantially different from the estimate of effect.

Open in a new tab

^a The corresponding likelihood of obtaining employment with intervention (and its 95% CI) is based on the assumed likelihood with the control intervention (= median likelihood across studies) and the relative effect of the intervention (and its 95% CI).  ^b Number of participants in direct comparison only.

^c We did not downgrade because of reporting bias as insufficient studies contributed to network treatment estimates to allow us to draw meaningful conclusions.

¹ We downgraded one level due to study limitations (majority moderate risk of bias studies).

² We downgraded one level due to study limitations (majority moderate risk of bias studies) and one level due to inconsistency (predictive interval for intervention effect includes effect that would have different interpretation and loop inconsistency).

³ We downgraded one level due to study limitations (majority moderate risk of bias studies), one level because of inconsistency (predictive interval for intervention effect includes effect that would have different interpretations) and one level for imprecision (CIs include values favouring either intervention).

⁴ We downgraded one level due to study limitations (majority moderate risk of bias studies) and one level because of imprecision (CIs include values favouring either intervention).

⁵ We downgraded two levels due to study limitations (majority high risk of bias studies).

⁶ We downgraded one level due to study limitations (majority moderate risk of bias studies).

⁷ We downgraded one level due to study limitations (majority moderate risk of bias studies) and one level because of imprecision (confidence intervals include values favouring either intervention).

⁸ We downgraded one level due to study limitations (majority moderate risk of bias studies) and one level because ofinconsistency (moderate level of heterogeneity).

⁹ We downgraded one level due to study limitations (majority moderate risk of bias studies), one level due to inconsistency (predictive interval for intervention effect includes effect that would have different interpretation and loop inconsistency) and one level because of detected publication bias (small study effects).

¹⁰ We downgraded one level due to study limitations (majority moderate risk of bias studies) and one level because of imprecision (confidence intervals include values favouring either intervention).    CI: confidence interval  RR: risk ratio

Background

Description of the condition

Mental illness is responsible for a significant loss of potential labour supply, high rates of unemployment, and a high incidence of sickness absence and reduced productivity at work (OECD 2012). Today in OECD countries, between one‐third and one‐half of all new disability benefit claims are for reasons of mental health, and among young adults that proportion goes up to over 70% (OECD 2012). Among people with severe mental illness the rates of work disability and unemployment are even higher. In the USA and the UK the employment rates for severely mentally ill people are reported to be less than 20% (Marwaha 2007; Salkever 2007). However, many people with severe mental illness do often have a desire to obtain some form of employment or participation in society (Hatfield 1992; McQuilken 2003; Mueser 2001).

There has been lack of consensus about a specific definition of severe mental illness (Delespaul 2013; Ruggeri 2000). Generally, severe mental illness is defined by three tangible indicators: diagnosis, disability and duration. The predominant diagnoses are psychotic disorders such as schizophrenia and schizoaffective disorder, bipolar disorders and major depression with psychotic features. Other psychiatric diagnoses such as personality disorders and severe concurrent diagnoses (psychiatric diagnosis with substance abuse) are sometimes included within the category of severe mental illness. The duration of the disorder suggests a persistence of mental health problems over time (e.g. frequency and intensity of use of psychiatric services). People with severe mental illness experience difficulties with functioning in one or more areas of daily life (Corbière 2013).

It's well known that work contributes to the quality of life of people in general and also for those with severe mental illness (Gold 2014; Lötters 2013; Schuring 2013). Competitive employment offers multiple advantages such as enhancing income, increasing self‐esteem, developing social skills, improvement of symptoms, decreasing number of hospital admissions and de‐stigmatisation (Bond 2001a; Burns 2009; Corbière 2009; Gold 2014; Mueser 2014; Perkins 2009), whereas unemployment can lead to social isolation with subsequent economic and social deprivation, which further reduces the probability of obtaining a job (Carlier 2013). Competitive employment means work in the competitive labour market that is performed on a full‐time or part‐time basis in an integrated setting; and for which an individual is compensated at or above the minimum wage, but not less than the customary or usual wage paid by the employer for the same or similar work performed by individuals who are not disabled. An integrated setting means a setting typically found in the community in which an individual with the most severe disabilities interacts with non‐disabled individuals (34 C.F.R. § 363.3(b) 2015).

Description of the intervention

In the past, people with severe mental illness were treated for long periods, or their whole lives, in hospital settings. Since the 1970s, a lot has changed. People with severe mental illness are increasingly living in the community supported by multidisciplinary community treatment teams, such as assertive community treatment or intensive case management. These are intensive mental health programme models that provide clinical and case management services. These programmes have substantially reduced psychiatric hospital use, increased housing stability, and moderately improved symptoms and subjective quality of life (Dieterich 2010; Marshall 1998). Along with this evolution in psychiatric care, the perspective on vocational participation has also changed. In the last decades, many vocational rehabilitation programmes have been developed and evaluated for people with severe mental illness. Vocational rehabilitation programmes are designed to help people with disabilities to obtain and maintain employment. Initially, people with severe mental illness, who had a desire to work, were placed in sheltered employment or enrolled in prevocational training and volunteer work before competitive employment. Competitive employment was considered to be too stressful. Caretakers preferred a stepwise approach so people could acquire certain skills before involvement in competitive employment.

In the mid 1980s, a new vocational rehabilitation approach emerged, known as supported employment (Crowther 2001). Supported employment emphasises a rapid search for a competitive job, with ongoing support provided as needed to get and keep the job (Drake 1999a). This method seems to be more effective in employment outcomes compared to prevocational training (Crowther 2001; Kinoshita 2013), however this intervention is not yet in wide use (OECD 2012). An important reason for this could be that the widespread implementation encounters too many financial or organisational barriers (Bond 2012a). Over the last couple of years, there has been a growing interest in strengthening supported employment with other prevocational skills training programmes such as social skills training. This method is called augmented supported employment or integrated supported employment (Boycott 2012).

At the moment the following interventions are in use.

Prevocational training

Prevocational training is a stepwise approach in which participants get trained before being employed. This approach is also called 'train, then place' or 'traditional vocational rehabilitation'. These programmes often use training classes, workshops, assessments or counselling. Training is provided in generic work skills or personal development such as self‐esteem, assertiveness and stress management (Corrigan 2001; Loveland 2007). There are also specific training programmes that focus on improvement of social or cognitive skills (Corbière 2009).

Social skills training

Social skills training consists of behavioural training focused on specific situations, problems, and activities. The ultimate goal of this training is the generalisation of learned skills to community‐based activities with improved functioning. Social skills training utilises principles from learning theory to improve social functioning by working with people to remediate problems in activities of daily living, leisure, relationships, or employment. There are two forms of social skills training: the basic model and the social problem‐solving model. In the basic model, complex social repertoires are broken down into simpler steps, subjected to corrective learning, practised through role playing, and applied in natural settings. The social problem‐solving model focuses on improving impairments in information processing that are assumed to be the cause of social skills deficits (Bellack 1993). In this model, social skills training is used to increase skills acquisition and reduce psychiatric symptoms in people with severe mental illness. Examples of social skills that are trained are assertiveness, use and interpretation of verbal and non‐verbal communication, and daily living skills (Dilk 1996; Kopelowicz 2006; Kurtz 2008).

Cognitive training

Cognitive functioning is often impaired in people with severe mental illness and is associated with poor vocational functioning (McGurk 2014). The terms 'cognitive training', 'cognitive remediation' and 'cognitive rehabilitation' are used both interchangeably and inconsistently in the literature and in clinical practice. These interventions are based on behavioural training and aim to improve cognitive processes (attention, memory, executive function, social cognition or meta‐cognition) with the goal of durability and generalisation. This can positively affect community functioning (Keshavan 2014; Wykes 2011).

Transitional employment

Transitional employment refers to segregated programmes designed to help individuals with disabilities who are viewed as not (yet) capable of working in a competitive employment setting. Usually transitional employment programmes are run by non‐profit organisations that receive funding through state and federal sources (Boardman 2003; Krainski 2013). Transitional employment can also be used as a first step to more gainful forms of employment.

Sheltered workshop

A transitional sheltered workshop refers to a workplace that provides a segregated working environment where people with a mental or physical disability can acquire job skills and vocational experience. It is hoped that this training and experience will assist them to acquire the skills necessary for competitive employment. There are also extended sheltered workshop programmes, typically designed to be long‐term placements for individuals who are expected not to be able to work in the community. Sheltered workshops are authorised to employ people with disabilities at sub‐minimum wages (Gervey 1994; Hsu 2009; Migliore 2010).

Social enterprise

A social enterprise/firm is a semi‐commercial business that offers paid employment at competitive rates for people who have difficulty integrating into the normal labour force (Boardman 2003; Gilbert 2013; Latimer 2005). Some of them are consumer‐run businesses (Latimer 2005). In an integrated working environment groups of clients are trained and supervised among both disabled and non‐disabled workers (Corbière 2009). A recent survey in the UK showed that over 50% of the employees were diagnosed with schizophrenia or bipolar disorder (Gilbert 2013). People with mental illness experience social enterprises as providing a flexible environment that promotes feelings of belonging, success, competence and individuality (Svanberg 2010). Through working in a social enterprise people can gain access to other more rewarding job opportunities in the labour market but can also increase psychosocial outcomes (Savio 1993; Vilotti 2014).

The Clubhouse model

A clubhouse is a building run by clients with severe mental illness and staff, where clients meet for social activity, mutual support and graded work experience. There are now approximately 300 clubhouses in various countries around the world, many of which are accredited by the International Center for Clubhouse Development (ICCD). The Clubhouse approach involves a period of preparation before clients attempt to return to competitive employment. This period of preparation consists of two stages: the work‐ordered day and transitional employment (Beard 1982). The work‐ordered day refers to a process whereby clients join work crews (working side by side with staff and other clients) that take responsibility for managing and maintaining the clubhouse. Work crews are a means to prepare for the next stage called transitional employment (Bond 1999; Norman 2006). Clients are discouraged from seeking competitive employment until they have achieved success in transitional employment, and are free to return to work crews at any time (Bilby 1992).

Supported employment

Supported employment programmes attempt to help adults with severe mental illness obtain competitive employment quickly and provide them with ongoing support to maintain employment (Bond 2001a).

Individual placement and support

The most clearly described and widely researched supported employment model is Individual Placement and Support (IPS). In this model, help is provided with looking for a job and, once employed, support is provided indefinitely. The services are integrated with mental health treatment services.

Individual placement and support is based on eight principles: (1) focus on competitive employment outcomes, (2) zero exclusion: open to anyone with severe mental illness who wants to work, (3) rapid job search, (4) attention to client preferences in services and job searches, (5) employment specialists systematically develop relationships with employers based upon client preferences, (6) time‐unlimited and individualised supports, (7) employment services are integrated with mental health treatment services, and (8) clients receive personalised benefit counselling (Becker 1993; Drake 2012).

Bond and colleagues studied the effectiveness of fidelity to the key principles of IPS (see above), and found evidence for the contribution of all these principles in helping people obtain and retain work (Bond 1999; Bond 2004).

Augmented supported employment

Augmented supported employment is supported employment augmented with other interventions, which may further increase employment outcomes. Any type of intervention can be used in combination with augmented supported employment; for example cognitive skills training with supported employment (Loveland 2007; McGurk 2004; Tsang 2009).

Psychiatric care only

Psychiatric care only is defined as usual psychiatric care for individuals with severe mental illness without any specific vocational component. Usually psychiatric care includes medication, supportive psychotherapy and case management. A well‐studied specific model of psychiatric care is assertive community treatment (ACT).

Assertive community treatment

ACT is a 24‐hour team‐based approach that provides clinical and case management to individuals with severe mental illness in the community. This multidisciplinary team consists of mental health care professionals such as case managers, a psychiatrist, psychiatric nurses, social workers and occupational therapists. The caseload is low, which enables intensive and frequent contacts. ACT teams practise 'assertive outreach', meaning that they continue to contact and offer services to reluctant or uncooperative people. They also place particular emphasis on medication compliance. In addition, ACT teams also provide help with housing, finances, activities of daily living, interpersonal relationships and employment (Bond 2001b; Stein 1998).

A number of studies, including meta‐analysis, demonstrate significant advantages of assertive community treatment in reducing hospital admissions, improving psychiatric symptoms and quality of life and increasing independent living (Dieterich 2010; Marshall 1998).

How the intervention might work

Prevocational training assumes that people with severe mental illness need to learn certain skills before they can hold a competitive job. In a protective environment, and in a stepwise way, people with severe mental illness are gradually exposed to 'normal' working conditions and routines. For some people, working in a competitive job is not possible (or not preferred by people with a severe mental illness) and working in a sheltered workplace may be their best possible option. These types of interventions focus on helping and empowering the individual.

In contrast, supported employment stands for rapid job search without extended preparation, but with prolonged intensive supervision on the job if needed. Another key component of supported employment is the integration of employment services and a mental health treatment team (Bond 2004). Therefore, it is also important that the mental health professionals agree on work rehabilitation as a treatment outcome. In addition, the job coach has an important role by empowering the employer in providing a healthy and stimulating workplace for the individual.

Although supported employment seems to be more effective than prevocational training in obtaining and maintaining competitive employment (Crowther 2001; Kinoshita 2013), there still seems to be room for improvement. A variety of factors, such as cognitive impairment or social difficulties, have been identified as contributing to brief job tenure and unsuccessful job terminations (Becker 1998; McGurk 2004). Consequently there is a growing interest in combining supported employment with other vocational interventions, such as social skills training or cognitive training (Bell 2008a; McGurk 2005; Mueser 2005; Tsang 2009). A certain number of people with severe mental illness may need a vocational rehabilitation process that combines elements from different types of interventions, focusing both on empowering the individual and on the future employer and working environment.

From research in occupational health settings, we know that a case manager who functions as an intermediary between the curative treatment setting and the work setting can decrease time to return to work (Schandelmaier 2012). This case manager may be a return‐to‐work co‐ordinator, an employment specialist, an occupational health specialist or the employer himself. It is possible that this element of case management as part of supported employment is the most effective aspect of this intervention. Corbière 2014 showed that SE employment specialists who use a client‐centred approach and have good relationships with employers and supervisors, have more vocational successes. A more recent study again confirmed that employment specialist skills are important to predict job acquisition (Corbière 2016).

Why it is important to do this review

Several systematic reviews have compared supported employment to one or more forms of prevocational training (e.g. Crowther 2001; Kinoshita 2013; Twamley 2003). The conclusions of these reviews suggest that supported employment may be more effective in obtaining competitive employment and may also increase length of employment. However, in the most recent systematic review (Kinoshita 2013), the authors considered the quality of evidence as very low, due to the small number of studies that contributed to the primary outcome (days in competitive employment). A large amount of data were considerably skewed, and therefore the authors excluded them from their meta‐analysis. There also appeared to be an overall high risk of bias in the individual studies.

In addition, not all forms of supported employment and prevocational training were compared, and therefore it remains somewhat unclear which particular form of supported employment or prevocational training actually has the largest effect on obtaining and maintaining work. Furthermore, not all types of interventions were directly compared with each other, and we do not know what the most effective components of these sometimes complex interventions are. Recently, new studies regarding supported employment enhanced with other prevocational interventions have been published but these results have not yet been included in the existing systematic reviews. It would be interesting to compare these results to those obtained with other types of interventions.

A network meta‐analysis enables us to perform direct and indirect comparisons between all types of interventions, and this may help clarify which components are particularly effective. We aim to present a ranking of these various types of vocational rehabilitation interventions based on their effectiveness. This ranking would be very helpful for mental and occupational healthcare professionals and policymakers interested in supporting people with severe mental illness to obtain and also to maintain employment.

Objectives

Methods

Criteria for considering studies for this review

Types of studies

We included randomised controlled trials (RCTs), including cluster‐randomised controlled trials, that assessed the effects of vocational rehabilitation interventions in people with severe mental illness. We excluded quasi‐experimental studies.

Types of participants

We included trials with adults aged between 18 and 70 years who had been diagnosed with severe mental illness. We defined severe mental illness as schizophrenia or other psychotic disorders, bipolar disorder, depression with psychotic features or other long‐lasting psychiatric disorders, with a disability in social functioning or participating in society, such as personality disorder, severe anxiety disorder, post‐traumatic stress disorder, major depression or autism with a duration of at least two years. Study participants had to be unemployed due to severe mental illness.

We excluded data from analyses where participants had a problem with substance abuse without any other mental disorder, or they had mental retardation, dementia, other neurocognitive disorders or terminal illness.

Types of interventions

We included trials of all types of vocational rehabilitation compared to each other or to no intervention or psychiatric care only.

We used the following classification of interventions and subgroups:

Prevocational training

Job‐related skills training
Symptom‐related skills training
1. Cognitive training
2. Social skills training

Transitional employment

Sheltered workshop
Social enterprise
Clubhouse model

Supported employment

Low‐fidelity IPS/not IPS
High‐fidelity IPS

Augmented supported employment

Supported employment + job‐related skills training
Supported employment + symptom‐related skills training
Supported employment + sheltered employment

Psychiatric care only

Types of outcome measures

We included studies only if they measured the primary outcome: percentage or number of participants who obtained competitive employment.

Primary outcomes

Percentage or number of participants who obtained competitive employment

The primary outcome of our review was obtaining competitive employment. This means work in the competitive labour market for which an individual is compensated at or above minimum wage.

We included all follow‐up times and categorised these as short‐term follow‐up if less than or up to 12 months, and long‐term follow‐up if longer than 12 months.

Secondary outcomes

Employment

Number of weeks in competitive employment
Number of days to first competitive employment
Percentage of participants who obtained non‐competitive employment (such as employment in a sheltered workplace or volunteer work)

Clinical outcomes

Quality of life (e.g. QOLI) (Lehman 1988)
Mental health (psychiatric symptoms) (e.g. PANSS) (Kay 1987)

Adverse events

Dropouts
Hospital admissions

Search methods for identification of studies

Electronic searches

First, we searched the following electronic databases from 1970 to 11 November 2016 to identify potentially relevant systematic reviews. We used comprehensive search strategies to find the eligible RCTs.

The Cochrane Central Register of Controlled Trials (CENTRAL; 2016 issue 11) via the Cochrane Library
MEDLINE (PubMed)
Embase
PsycINFO
CINAHL

Second, we searched for additional RCTs, which were not yet included in systematic reviews. We searched the same databases.

We developed two electronic search strategies by combining search words for the concepts 'mental disorder', 'return to work' and 'systematic reviews' or 'randomised controlled trial' (Appendix 1; Appendix 2; Appendix 3; Appendix 5; Appendix 6; Appendix 7; Appendix 8; Appendix 9). We searched with these concepts for reviews and additional RCTs.

We used PubMed's 'My NCBI' (National Center for Biotechnology Information) email alert service for identification of newly published systematic reviews and RCTs using a basic search strategy.

Searching other resources

We checked the reference lists of all identified studies for additional potentially relevant studies. Additionally, we consulted domain experts with years of experience in vocational rehabilitation for people with severe mental illness, to identify unpublished materials.

Data collection and analysis

Selection of studies

First, we focused on the selection of systematic reviews and then we continued with the selection of additional RCTs. Two authors (YS, FS) independently screened the titles and abstracts of publications identified by both search strategies. We discarded all studies that were not applicable according to our inclusion criteria. We then obtained the full text of the remaining references. Two authors (YS, FS) independently decided whether the reviews and RCTs met the inclusion criteria and classified the different types of interventions. We resolved disagreements through discussion. In case of persistent disagreement we consulted a third author (JM).

Data extraction and management

Two authors (YS, JM) extracted data for our meta‐analyses and network meta‐analyses from the individual RCTs that were included in the systematic reviews, and from the additional RCTs that we included. Two authors (YS, JM) independently extracted characteristics and outcome data of the included studies. We resolved disagreements through discussion or with assistance from a third author (FS) when necessary. We used a data collection form that was specifically designed and piloted by the author team. We extracted the following study characteristics:

Methods: study design, total duration of study, study location and setting, year of publication
Participants: number of participants, diagnosis, duration of mental illness, inclusion and exclusion criteria, gender, mean age, ethnicity, work history, disability benefits.
Interventions: description of intervention, comparison, duration, intensity.
Outcomes: description of primary and secondary outcomes, moment of measurements.

Assessment of risk of bias in included studies

We used the Cochrane tool for assessing risk of bias to assess the methodological quality of the included trials (Higgins 2011a; Appendix 10).

We assessed the following domains according to this tool:

Sequence generation (selection bias)
Allocation concealment (selection bias)
Blinding of participants and personnel (performance bias)
Blinding of outcome assessment (detection bias)
Incomplete outcome data (attrition bias)
Selective outcome reporting (reporting bias)
Other potential sources of bias

Each of the domains were scored as 'high', 'low' or 'unclear' risk of bias, following criteria outlined in Chapter 8 of the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2011a). We summarised the risk of bias judgements across different studies for each of the domains listed. We conducted sensitivity analyses by excluding studies we judged to have a high risk of bias in any of the domains listed above.

In case of cluster‐RCTs, we addressed the six components of the 'Risk of bias' tool as well as recruitment bias, baseline imbalance, loss of clusters, incorrect analysis and comparability with individually randomised trials.

For RCTs that had previously been included in systematic reviews, one review author (JM) reassessed the risk of bias and ensured that the results agreed with those published. In case of disagreement, JM discussed the risk of bias with another author (YS). We contacted the original author of the systematic review if disagreement persisted.

Two authors (YS, JvM) independently assessed the risk of bias of the RCTs that were not included in an existing systematic review. In case of persistent disagreement, we consulted a third review author (FS). If information was absent for evaluation of the methodological criteria, we contacted the authors of the study with a request to provide additional information.

Measures of treatment effect

We expressed dichotomous outcome data as risk ratios (RR) with their 95% confidence interval (CI). Additionaly, we calculated the corresponding risks of the interventions for the primary outcome. For continuous data, we used the mean difference (MD) when outcome measurements were made on the same scale. If the same outcome was measured with different scales, we calculated the standardised mean difference (SMD) with its 95% CI.

Unit of analysis issues

Cluster‐RCTs

With cluster‐RCTs we used the group estimates taking into account the cluster randomisation. Studies in which clusters of individuals were randomised to groups, but where the intervention was intended to work at the level of the individual, were analysed taking into account the intra‐cluster correlation coefficient (ICC), as was explained in the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2011b). When the authors of cluster‐randomised trials did not report ICCs, we assumed an ICC of 0.1 (Higgins 2011b).

Multi‐arm studies

With multi‐arm studies we used the data from all comparisons.

Dealing with missing data

We contacted trial authors to obtain missing data. Where this was not possible, or the missing data could have led to serious bias, we explored the impact of including these studies in the overall assessment of results by a sensitivity analysis.

If numerical outcome data such as standard deviations (SDs) or correlation coefficients were missing and they could not be obtained from the study authors, we calculated them from other available statistics such as P values, according to the methods described in the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2011b).

We reported information regarding loss to follow‐up, and we assessed this as a potential risk of bias.

Assessment of heterogeneity

In the standard pairwise meta‐analyses we estimated different heterogeneity variances for each pairwise comparison. We assessed statistically the presence of heterogeneity within each pairwise comparison using the I² statistic (Higgins 2003), where an I² value of 25% to 49% indicates a low degree of heterogeneity, 50% to 75% a moderate degree of heterogeneity and more than 75% indicates a high degree of heterogeneity.

In the network meta‐analyses we assumed a common estimate for the heterogeneity variance across the different comparisons. The assessment of statistical heterogeneity in the entire network is based on the magnitude of the heterogeneity variance parameter estimated from the network meta‐analysis models. We used the Tau² statistic to assess heterogeneity within the comparisons. A Tau² value greater than 1 suggests presence of substantial statistical heterogeneity. We assessed the assumption of transitivity by comparing the distribution of the potential effect modifiers age, gender and working history across the comparisons.

Assessment of reporting biases

We employed a comparison‐adjusted funnel plot for the detection of small study effects (Chaimani 2013). Assymetry in the funnel plot can indicate the presence of small study effects, which can be a result of publication bias.

Data synthesis

Relative treatment effects

We extracted the risk ratio (RR) and reported the RR and the corresponding risks for the primary outcome: number of participants in competitive employment. For one other outcome measure (the number of days or weeks in competitive employment), we used the pooled mean differences (MDs). To avoid confusion about the number of days or weeks, we recalculated this considering the number of hours worked per week. We calculated the standardised mean difference (SMD) as a summary statistic when the concepts of being in competitive employment were the same but authors of studies had used different measurement scales.

Where comparable data and outcomes existed for different interventions, we performed direct and indirect comparisons using a network analysis and multiple treatments meta‐analysis (White 2011). A network meta‐analysis differs from standard pairwise meta‐analysIs primarily because it uses information across available comparisons to estimate indirect pairwise comparisons. We have presented results from network meta‐analyses as summary relative effect sizes (MD or SMD) for each possible pair of treatments.

Methods for direct and indirect or mixed treatment comparisons

In the direct comparisons, we pooled data from studies we judged to be clinically homogeneous using Review Manager 5 (RevMan 5) software (RevMan 2014). We performed standard pairwise meta‐analysis for every comparison that contained at least two studies. We used a random‐effects model if studies had high statistical heterogeneity (I² > 75%); otherwise we used a fixed‐effect model.

In the indirect and mixed comparisons, we performed network meta‐analyses in STATA version 13 using the mvmeta command (White 2012) and self‐programmed STATA routines available at http://www.mtm.uoi.gr.

Assessment of statistical inconsistency in network meta‐analysis

To evaluate the presence of inconsistency locally we used the loop‐specific approach. This method evaluates the consistency assumption in each closed loop of the network separately as difference between direct and indirect estimates for a specific comparison in the loop (inconsistency factor). Then, we used the magnitude of the inconsistency factors and their 95% confidence intervals (CIs) to make inferences about the presence of inconsistency in each loop. We assumed a common heterogeneity estimate within each loop. We present this approach graphically in a forest plot using the ifplot command in STATA.

To check the assumption of consistency in the entire network we used the design‐by‐treatment model as described by Higgins 2012. This method accounts for different sources of inconsistency that can occur when studies with different designs (two‐arm trials versus three‐arm trials) give different results and when there is disagreement between direct and indirect evidence. Using this approach we made inferences about the presence of inconsistency from any source in the entire network based on a Chi² test. We performed the design‐by‐treatment model in STATA using the mvmeta command. Inconsistency and heterogeneity are interwoven: to distinguish between these two sources of variability we employed the I² statistic for inconsistency, as it measures the percentage of variability that cannot be attributed to random error or heterogeneity (within comparison variability).

Relative treatment ranking

We also estimated the ranking probabilities for all treatments at each possible rank for each intervention. Then, we obtained hierarchy using the surface under the cumulative ranking curve (SUCRA) and mean ranks (Salanti 2011). SUCRA can also be expressed as a percentage of a treatment that can be ranked first without uncertainty. We performed the SUCRA curves and percentages in STATA using SUCRA commands.

Subgroup analysis and investigation of heterogeneity

We performed an intervention subgroup analysis to reduce the possible differences in effectiveness between interventions. Furthermore, we analysed the data to see whether the three possible effect modifiers (age, gender and working history) actually influenced the difference in effect within the network meta‐analysis, focusing on exploring potential inconsistency and heterogeneity.

Sensitivity analysis

If sufficient studies were available, we assessed the effect of excluding studies from the analysis that we had judged to have a high risk of bias. We also checked what the effect was of assumptions that we had made about transitivity.

Quality of the evidence

We used the GRADE approach (GRADE Working Group 2013) and the recommendations for network meta‐analyses of Salanti 2014 to assess the quality of evidence for each important comparison. GRADE is used in Cochrane systematic reviews to grade and report quality of evidence within 'Summary of findings' tables. We described the risk of bias as it was assessed within the included reviews. Two authors (YS, FS) assessed the quality criteria independently and resolved any disagreements by discussion.

Results

Description of studies

See: Characteristics of included studies, Characteristics of excluded studies and Characteristics of ongoing studies.

Results of the search

First we searched for relevant systematic reviews to find RCTs. We identified a total of 1885 records and assessed 46 full‐text reviews. Out of nine systematic reviews (Almerie 2015; Arbesman 2011; Bond 2015a; Bouvet 2014; Crowther 2001; Dieterich 2010; Heffernan 2011; Kinoshita 2013; Twamley 2003) we extracted 29 RCTs that fulfilled our eligibility criteria (see Criteria for considering studies for this review). Additionally, we found 7650 records, of which we assessed 88 full‐text RCTs. In total 19 RCTs were also found to be eligible, leading to 48 included RCTs, of which 42 (N = 6712) could be included in the network meta‐analyses and/or meta‐analyses. See Figure 1 for a PRISMA study flow diagram. See Characteristics of excluded studies for the reasons why we excluded studies. Two review authors (YS and FS) checked the eligibility of the identified RCTs. We updated the search on 11 November 2016. Newly published trials will be included in the next version (see Studies awaiting classification). Two new systematic reviews were also identified (Chan 2015; Modini 2016).

Included studies

See Table 2 and Table 3 for an overview of all included RCTs, their main characteristics, intervention classifications, outcomes and contribution to the analyses.

1. Descriptive details of included studies.

Study	Country	Follow‐up^a	N	Mean age	Male participants	Diagnosis (majority)	Working history (majority)
Au 2015	China	short	90	36	63%	Psychotic disorder	yes
Beard 1963	USA	short	212	N/A	60%	Psychotic disorder	N/A
Becker 1967	USA	short	50	46	N/A	Psychotic disorder	yes
Bejerholm 2015	Sweden	long	120	38	56%	Psychotic disorder	yes
Blankertz 1996	USA	short	122	36	64%	Psychotic disorder	yes
Bond 1986	USA	long	131	25	69%	Psychotic disorder	yes
Bond 1995	USA	short	86	35	51%	Psychotic disorder	yes
Bond 2007	USA	long	200	39	64%	Psychotic disorder	no
Bond 2015b	USA	short	90	44	79%	Psychotic disorder	yes
Burns 2007	Europe (UK, Italy, Germany, Netherlands, Bulgaria, Switzerland)	long	312	38	60%	Psychotic disorder	yes
Burns 2015	UK	long	123	38	59%	Psychotic disorder	yes
Chandler 1996	USA	long	256	N/A	43%	Psychotic disorder	N/A
Craig 2014	UK	short	159	24	73%	Psychotic disorder	yes
Dincin 1982	USA	short	132	25	53%	Psychotic disorder	N/A
Drake 1996	USA	long	143	37	48%	Psychotic disorder	N/A
Drake 1999b	USA	long	152	39	39%	Psychotic disorder	N/A
Drake 2013	USA	long	2238	44	47%	Affective disorder	N/A
Drebing 2005	USA	short	21	46	95%	Affective disorder + substance dependence	yes
Drebing 2007	USA	short	100	46	99%	Affective disorder + substance dependence	yes
Eack 2009	USA	long	58	26	69%	Psychotic disorder	N/A
Gervey 1994	USA	short	34	19	67%	N/A	no
Gold 2006	USA	long	143	N/A	38%	Psychotic disorder	yes
Hoffmann 2012	Switzerland	long	100	34	65%	Affective disorder	yes
Howard 2010	UK	long	219	38	67%	Psychotic disorder	yes
Killackey 2008	Australia	short	41	21	81%	Psychotic disorder	yes
Killackey 2014	Australia	short	146	20	67%	Psychotic disorder	yes
Latimer 2006	Canada	short	150	40	62%	Psychotic disorder	yes
Lecomte 2014	Canada	short	24	32	71%	Psychotic disorder	N/A
Lehman 2002	USA	long	219	42	57%	Psychotic disorder	yes
McFarlane 1996	USA	long	68	30	65%	Psychotic disorder	N/A
McFarlane 2000	USA	long	69	33	70%	Psychotic disorder	N/A
McGurk 2007	USA	long	48	38	55%	Psychotic disorder	yes
McGurk 2009	USA	long	34	44	59%	Psychotic disorder	yes
Michon 2014	Netherlands	long	151	35	74%	Psychotic disorder	yes
Mueser 2004	USA	long	135	41	61%	Psychotic disorder	yes
Nuechterlein 2012	USA	long	69	25	67%	Psychotic disorder	N/A
O'Brien 2003	UK	short	1037	N/A	55%	Psychotic disorder	yes
Oshima 2014	Japan	short	37	41	49%	N/A	yes
Penk 2010	USA	short	89	45	100%	Affective disorder +substance abuse/dependence	yes
Schonebaum 2006	USA	long	177	38	55%	Psychotic disorder	yes
Tsang 2001	China	short	97	36	62%	Psychotic disorder	yes
Tsang 2010	China	long	189	35	49%	Psychotic disorder	yes
Twamley 2012a	USA	short	58	51	64%	Psychotic disorder	yes
Viering 2015	Switzerland	long	183	43	47%	Affective disorder	yes
Waghorn 2014	Australia	short	208	32	69%	Psychotic disorder	N/A
Walker 1969	USA	short	28	N/A	96%	Psychotic disorder	N/A
Wong 2008	China	long	92	34	60%	Psychotic disorder	N/A
Xiang 2007	China	long	103	38.6	47%	Psychotic disorder	N/A

Open in a new tab

^aFollow‐up: short ≤ 1 year; long > 1 year.

^bSecondary outcomes:

1 = maintaining employment  2 = obtaining non‐competitive employment  3 = days to first competitive employment  4 = mental health  5 = quality of life  6 = dropouts  7 = hospital admissions.

2. Comparisons and outcomes in included studies.

Study	Comparison intervention main group	Comparison intervention subgroups	Secondary outcomes^b	Included in meta‐analysis	Included in network met‐analysis
Au 2015	SE+ vs SE+	SE+ symp vs SE+ symp	1, 4, 5, 6	no	no
Beard 1963	TE vs psych care	CT vs psych care	7	yes	no
Becker 1967	TE vs psych care	SWS vs psych care	2, 7, 6	yes	no
Bejerholm 2015	SE vs PVT	hf IPS vs job skills training	1, 2, 3,5 6,	yes	yes
Blankertz 1996	PVT vs psych care	Job skills training vs psych care	2, 6	yes	no
Bond 1986	TE vs TE	Not classified CH accelerated vs gradual	2, 6, 7	no	no
Bond 1995	SE+ vs SE	SE+job skills training vs lfIPS	1, 2, 7	yes	no
Bond 2007	SE vs TE	hf IPS vs CH	1,2,3,4,5,6	yes	yes
Bond 2015b	SE vs PVT	hf IPS vs job skills training	1, 2,, 6, 7	yes	no
Burns 2007	SE vs PVT	hf IPS vs job skills training	1, 3, 4, 5, 6, 7	yes	yes
Burns 2015	SE vs SE	hf IPS vs lf IPS	1, 3, 4,5, 6, 7,	no	yes (sub)
Chandler 1996	SE+ vs psych care	SE+ACT vs ACT	2, 5, 6, 7	yes	no
Craig 2014	SE+ vs SE	SE+motivational interviewing vs hf IPS	1, 2, 6	yes	no
Dincin 1982	TE vs psych care	CH vs psych care care	, 6, 7	yes	no
Drake 1996	SE vs PVT	lf IPS vs job skills training	4, 5, 6	yes	yes
Drake 1999b	SE vs TE	hf IPS vs SWS	2, 3, 4, 5, 6	yes	yes
Drake 2013	SE vs psych	hf IPS vs psych	1, 2, 4, 5, 6	yes	yes
Drebing 2005	SE+ vs SE+	unclassified SE+TE+contingency management vs SE+TE	1, 6	no	no
Drebing 2007	SE+ vs SE+	unclassified SE+TE+contingency management vs SE+TE	1	no	no
Eack 2009	PVT vs psych care	CT vs psych care	4, 6	yes	yes
Gervey 1994	SE vs TE	lf IPS vs SWS	1	yes	no
Gold 2006	SE vs TE	hf IPS vs SWS	1,2,3,4,5,6	yes	yes
Hoffmann 2012	SE vs TE	hf IPS vs SWS	1, 2, 3, 4, 5, 6	yes	yes
Howard 2010	SE vs PVT	hf IPS vs job skills training	4, , 5, 6, 7	yes	yes
Killackey 2008	SE vs psych	hf IPS vs psych	1, 6	yes	no
Killackey 2014	SE vs psych	hf IPS vs psych	6	yes	no
Latimer 2006	SE vs TE	hf IPS vs SWS	1, 2, 3, 6	yes	no
Lecomte 2014	SE+ vs SE	SE+symp vs hfIPS	1, 2	yes	no
Lehman 2002	SE vs PVT	hf IPS vs job skills training	2, 6	yes	yes
McFarlane 1996	Psych care vs psych care	Not classified ACT+multifamily groups vs ACT+crisis family intervention	2, 4	no	no
McFarlane 2000	SE+ vs TE	ACT+SE vs SWS	1, 2	yes	yes
McGurk 2007	SE+ vs SE	SE+symp vs lf IPS	1, 4, 7, 6	yes	yes
McGurk 2009	SE+ vs SE	SE+symp vs lf IPS	1, 4	yes	yes
Michon 2014	SE vs PVT	hf IPS vs job skills training	1, 2, 3, 4, 5, 6, 7	yes	yes
Mueser 2004	SE vs TE	hf IPS vs CH	1, 2, 3, 4, 6	yes	yes
Nuechterlein 2012	SE+ vs PVT	SE+job vs SST	6	yes	yes
O'Brien 2003	SE vs psych care	lf IPS vs psych care	2, 6, 7	yes	no
Oshima 2014	SE vs PVT	hf IPS vs job skills training	1, 2, 6	yes	no
Penk 2010	TE vs PVT	SWS vs job skills training	1, 2, 3, 6	yes	no
Schonebaum 2006	SE+ vs SE+	ACT+SE vs SE+TE	1, 6	yes	yes(sub)
Tsang 2001	PVT vs psych care	SST vs psych care	none	yes	no
Tsang 2010	SE+ vs SE vs PVT	SE+symp vs hf IPS vs job skills training	1, 5, 6	yes	yes
Twamley 2012a	SE vs PVT	hf IPS vs job skills training	1, 2, 3, 6	yes	no
Viering 2015	SE vs PVT	lf IPS vs job skills training	6	yes	yes
Waghorn 2014	SE vs SE	lf IPS vs hfIPS	1, 6	no	no
Walker 1969	TE vs psych care	SWS vs psych care	1, 2, 7	yes	no
Wong 2008	SE vs PVT	hf IPS vs job skills training	1, 2, 3, 6	yes	yes
Xiang 2007	PVT vs psych care	SST vs psych care	4, 6, 7	yes	yes

Open in a new tab

(sub) = included in subgroup network meta‐analysis only.

ACT: assertive community treatment  CH: Clubhouse  CT: cognitive training  job: job related skills training  hf IPS: high‐fidelity Individual Placement and Support  lf IPS: low‐fidelity Individual Placement and Support  Psych care: psychiatric care only  PVT: prevocational training  SE: supported employment  SE+: augmented supported employment  SST: social skills training  SWS: sheltered workshops  Symp: symptom‐related skills training  TE: transitional employment

Design

All studies were longitudinal RCTs. Fourteen RCTs were multi centre trials (Au 2015; Bond 1995; Bond 2007; Bond 2015b; Burns 2007; Chandler 1996; Drake 1996; Drake 2013; Howard 2010; Michon 2014; Schonebaum 2006; Tsang 2001; Tsang 2010; Waghorn 2014) and two used a multi‐arm design (Mueser 2004; Tsang 2010).Three studies used a cluster‐randomised design (Craig 2014; O'Brien 2003; Tsang 2001). O'Brien 2003 reported an ICC of 0.00148, indicating a low design effect. The other two trials did not report an ICC. Therefore we adjusted the data to the possible design effect with an assumed ICC of 0.1. We used this equation: 1+(M‐1)ICC to calculate the design effect according to the average cluster size (M) and the (assumed) ICC. For dichotomous data we divided the number of participants and the number of events by the design effect. We only reduced the sample size for continuous data. This method is described in the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2011b).

Setting and time

The majority of included RCTs (N = 30) were performed in North America. Five studies had been conducted in China, four in the UK, three in Australia, two in Switzerland and one each in Japan, the Netherlands and Sweden. One study was a European collaboration between the UK, Italy, Germany, the Netherlands, Bulgaria and Switzerland. Four RCTs were performed at the same psychiatric rehabilitation centre (Threshold, Chicago, USA) (Bond 1986; Bond 2007; Bond 2015b; Dincin 1982). The follow‐up ranged from three months (Tsang 2001) up to five years (Hoffmann 2012). The majority of included RCTs (N = 37) were published the year 2000 or later (see Table 2).

Participants

The 48 RCTs included 8743 participants with an average of 182 participants per study. The smallest RCT consisted of 21 participants (Drebing 2005, a pilot study) and the largest included 2238 participants (Drake 2013). In 22 RCTs the majority of the participants were white. Of the other RCTs, 13 involved a majority of non‐white participants and 16 RCTs did not describe the ethnicity of the participants. Remarkably, in 10 of out of 30 US trials, the majority of participants were non‐white. The mean age of the participants was 36 years. In one RCT (Gervey 1994) the age of the participants was only 19 years and another RCT (Twamley 2012a) specifically focused on older adults (mean age 51 years). The majority (63%) of the participants were male. In four studies more than 90% of the participants were male (Drebing 2005; Drebing 2007; Penk 2010; Walker 1969). This could be explained by the fact that these participants were all veterans. Marital status was described in 26 trials and nearly all participants were single.

The great majority (N = 39) of the RCTs contained predominantly participants with a psychotic disorder (schizophrenia, schizoaffective or other psychotic disorders). In only four studies (Drake 2013; Drebing 2005; Drebing 2007; Penk 2010) was an affective disorder the main diagnosis of the included participants. In three out of these four trials the majority of participants were diagnosed with substance use disorder and comorbid affective disorder (dual diagnosis) (Drebing 2005; Drebing 2007; Penk 2010). Aside from these trials, in another three RCTs (Blankertz 1996; Killackey 2008; Lehman 2002) substance abuse was described by the majority of participants. Most RCTs did not mention substance abuse or excluded these participants. Four RCTs exclusively included young adults with first episode psychosis (Craig 2014; Killackey 2008; Killackey 2014; Nuechterlein 2012). Somatic comorbidity was outlined in 11 trials. In McGurk 2009 74% of the participants had one or more medical co‐morbidities, often hypertension or diabetes, while almost all participants (92%) in Killackey 2008 were free of medical illness. In the remaining nine studies medical illness/physical impairment that could prevent participation or return to work was an exclusion criteria (Becker 1967; Bejerholm 2015; Bond 2007; Bond 2015b; Craig 2014; Drake 1996; Latimer 2006; McFarlane 1996; Wong 2008).

Consistent with our eligibility criteria, almost all trials included only participants who were unemployed at baseline. We included four additional RCTs with a small percentage (between 8% and 27%) of participants who were partly employed at baseline, and were willing to obtain another job (Eack 2009; Killackey 2008; Killackey 2014; Viering 2015). In one study (Viering 2015) we could extract the data of only those participants who were fully unemployed.

Thirty‐three studies described the percentage of participants with a working history, in 31 of these studies the majority had worked in the past (see Table 2). Most participants in 13 studies had worked recently (in the past five years) (Bejerholm 2015; Burns 2007; Drebing 2005; Drebing 2007; Gold 2006; Howard 2010; Latimer 2006; Michon 2014; Mueser 2004; Penk 2010; Schonebaum 2006; Tsang 2001; Viering 2015). Interest in (competitive) employment was an eligibility criteria in 34 studies. The educational level of the participants can be classified as mainly secondary educated. In five RCTs (Becker 1967; Gervey 1994; Michon 2014; Tsang 2001; Viering 2015) the participants had received primary education, and in another five trials (Au 2015; Drake 1996; Hoffmann 2012; Killackey 2008; Latimer 2006) the educational level was tertiary. In 15 RCTs (Bond 1986; Bond 1995; Bond 2007; Bond 2015b; Chandler 1996; Drake 2013; Drebing 2007; Gold 2006; Hoffmann 2012; Killackey 2008; Lehman 2002; McFarlane 2000; Michon 2014; Viering 2015; Waghorn 2014), the majority of participants received a disability benefit; in two RCTs (Drake 2013; Viering 2015) receiving a disability benefit was a requirement for enrolment in the study. Most trials, though, did not report benefit or other financial support status. Interestingly, only one RCT primarily focused on participants with criminal justice involvement (Bond 2015b).

Interventions

The interventions of all studies could be classified in our predefined intervention main groups: supported employment, augmented supported employment, prevocational training, transitional employment and psychiatric care only. In five studies (Au 2015; Bond 1986; Bond 2015b; Drebing 2005; Drebing 2007) the intervention and control conditions were classified as the same main group and we could only classify the intervention further, as one of our predefined subgroups, in one of these studies (Burns 2015). Subgroup classification of the other four studies was not possible, because these interventions had too many components. All subgroup interventions were represented in the included studies except for social enterprise.

Figure 2 (main groups) and Figure 3 (subgroups) show the networks of evidence for the benefit (obtaining competitive employment) of the included interventions in the network meta‐analysis. Each line refers to the interventions that have been directly compared in studies. The thickness of the line is proportional to the number of participants included in the comparison and the width of each circle is proportional to the number of studies included in the comparison.

Network plot of direct comparisons of intervention main groups (long‐term follow‐up). **Psych care**: psychiatric care only; **PVT**: prevocational training; SE: supported employment; **SE+**: augmented supported employment; TE: transitional employment

Network plot of direct comparisons of intervention subgroups (long‐term follow‐up). CH: Clubhouse; CT: cognitive training; **hf IPS**: high‐fidelity Individual Placement and Support; **job** : job‐related skills training; **lf IPS**: low‐fidelity Individual Placement and Support; **Psych care**: psychiatric care only; **SE + ACT**: supported employment + assertive community treatment; **SE + job**: supported employment + job‐related skills training; **SE + symp**: supported employment + symptom‐related skills training; **SE + TE**: supported employment + transitional employment; **SST**: social skills training; **SWS**: sheltered workshops

Prevocational training

We classified prevocational training as the main component of the intervention or control condition in 17 RCTs. We found four trials that compared prevocational training to psychiatric care only (Blankertz 1996; Eack 2009; Tsang 2001; Xiang 2007). Blankertz 1996 used job‐related skills training, Eack 2009 used cognitive training, Tsang 2001 and Xiang 2007 used social skills training. The cognitive training in Eack 2009 consisted of three months of weekly, computer‐based neurocognitive training and social cognitive group training sessions. In Tsang 2001 the participants were engaged in 10, weekly group sessions. This was a three‐armed trial with two arms of social skills training with or without follow‐up contacts. Xiang 2007 applied a standardised social skills training programme, the Community Re‐entry Module, to facilitate the transition from hospital to community. This programme contained 16 group sessions.

Twelve RCTs compared SE to job‐related skills training (Bejerholm 2015; Bond 2015b; Burns 2007; Drake 1996; Howard 2010; Lehman 2002; Michon 2014; Tsang 2010; Twamley 2012a; Viering 2015; Wong 2008; Xiang 2007). In addition, Penk 2010 compared sheltered workshops to job‐related skills training and Nuechterlein 2012 compared SE plus job‐related skills training to social skills training. The interventions classified as job‐related skills training were very heterogeneous regarding the support that the participants received. This was a group of interventions/usual care using a stepwise approach, with components of job counselling/coaching and training sessions for job skills or other work‐related tasks such as job interviewing.

Transitional employment

In six RCTs transitional employment was the intervention of interest (Beard 1963; Becker 1967; Bond 1986; Dincin 1982; Penk 2010; Walker 1969) and in eight the control condition (Bond 2007; Drake 1999b; Gold 2006; Hoffmann 2012; Latimer 2006; McFarlane 2000; Mueser 2004; Oshima 2014), most frequently compared to supported employment. Nine RCTs described transitional employment as sheltered workshops (Beard 1963; Becker 1967; Dincin 1982 Drake 1999b; Gold 2006; Hoffmann 2012; Latimer 2006; McFarlane 2000; Penk 2010) and five RCTs used the Clubhouse model (Beard 1963; Bond 1986; Bond 2007; Dincin 1982; Mueser 2004) We did not find any trials that evaluated social enterprises.

Supported employment

The most common intervention was supported employment, as we found 30 RCTs that included supported employment as intervention or control condition, see Table 2. The majority (24 RCTs) used high‐fidelity individual placement and support (IPS), In two studies, both the intervention and control condition were supported employment but differed in IPS fidelity (Burns 2015; Waghorn 2014). In the other RCTs, supported employment was the intervention condition, apart form six RCTs that compared augmented supported employment to supported employment only (Bond 1995; Craig 2014; Lecomte 2014; McGurk 2007; McGurk 2009; Tsang 2010). Most included RCTs reported IPS fidelity scores or described the classification (most often as 'good'). Author used the IPS fidelity scale most (such as in Bond 1997), but some trials (Bejerholm 2015; Bond 2015b; Craig 2014) applied a newer edition (Bond 2012b), and Michon 2014 used the QSEIS (Bond 2002). However, some older trials reported no fidelity scores, probably because they were carried out before the development of the fidelity scale (Bond 1995; Chandler 1996; Drake 1996; Gervey 1994). Some newer RCTs (including augmented supported employment) did not use a fidelity scale either (Au 2015; Burns 2015; Killackey 2014; McGurk 2009; Nuechterlein 2012; O'Brien 2003; Schonebaum 2006). In these cases we (YS, FS) discussed fidelity classification (high or low) based on description of the intervention and fidelity in the reports.

Augmented supported employment

We identified 13 RCTs that combined supported employment with another intervention. Au 2015 and Tsang 2010 combined supported employment with social skills training. Tsang 2010, a three‐armed trial, compared supported employment augmented with social skills training, Supported employment only, and prevocational job‐related skills training. Au 2015 compared su[ported employment combined to social skills training and cognitive training, with supported employment combined with social skills training only. Au 2015 and Tsang 2010 described 10 sessions of work‐related social skills training, conducted prior to job search. Nuechterlein 2012 tested the Workplace Fundamental Module, in which the participants receive group skills training for a year integrated with IPS compared to a skills training only. Three other RCTs also focused on the addition of cognitive training or cognitive behavioural interventions (Lecomte 2014; McGurk 2007; McGurk 2009). The cognitive training in Au 2015 consisted of three sessions a week for three months, with visual‐based, computer‐assisted cognitive exercises by two cognitive remediation software systems (Strongarm and Captain's Log). McGurk 2007 and McGurk 2009 also used computer‐based cognitive exercises (Cogpack) for two to three sessions a week for 12 to 16 weeks. The cognitive therapist worked together with the employment specialist and advised about cognitive impairments and supports needed to enhance work performance. McGurk 2009 also used group sessions. Lecomte 2014 implemented eight sessions during one month of group cognitive behavioural therapy. Bond 1995 compared a period of prevocational job‐related skills training followed by supported employment to immediate enrolment in supported employment(without training).

Four studies (Chandler 1996; Gold 2006; McFarlane 2000; Schonebaum 2006) combined supported employment to assertive community treatment (ACT). Schonebaum 2006 compared supported employment plus Clubhouse model to supported employment plus ACT. McFarlane 2000 and Gold 2006 coupled ACT and supported employment and compared this to transitional employment. Chandler 1996 compared supported employment plus ACT to ACT only. Two studies (Drebing 2005; Drebing 2007), including one separate pilot study, focused on contingency management. They used incentives for taking steps towards obtaining and maintaining competitive employment and for abstinence from substance abuse. The participants (veterans) were placed in transitional employment, but this programme also included supported employment components. One study (Craig 2014) added motivational interviewing for care co‐ordinators to supported employment. Six studies delivered high‐fidelity IPS (Craig 2014; Gold 2006; Lecomte 2014; Nuechterlein 2012; Schonebaum 2006; Tsang 2010).

Psychiatric care only

In 13 RCTs the control group did not consist of specific vocational interventions (Beard 1963; Becker 1967; Blankertz 1996; Chandler 1996; Dincin 1982; Drake 2013; Eack 2009; Killackey 2008; Killackey 2014; O'Brien 2003; Tsang 2001; Walker 1969; Xiang 2007). In most of these trials the control condition was care as usual. One study (McFarlane 1996) compared ACT plus family psycho‐education group to ACT plus crisis family intervention. We did not include this trial in the analyses because we could not classify both arms in different intervention groups.

Outcomes

Primary outcome

All RCTs reported data for our primary outcome (percentage or number of participants who obtained competitive employment). The definition used was not always equal. Some trials required a period of time being in competitive employment or a minimum number of hours worked a week, or both, before they counted it as a successful result (Au 2015; Bejerholm 2015; Bond 1986; Drebing 2007; Gervey 1994; Hoffmann 2012; Oshima 2014; Penk 2010; Tsang 2010; Twamley 2012a; Viering 2015; Wong 2008; Xiang 2007). This ranged from a minimum requirement of five days' Hoffmann 2012 to three months' (Xiang 2007) consecutive working and from five hours' (Oshima 2014) to 20 hours' (Tsang 2010; Drebing 2007) working a week. Moreover, Tsang 2010 also required two months of working.

In the majority of the RCTs the type of job counted as competitive employment was well described, in agreement with our definition. We also identified four RCTs with less clear definitions. Beard 1963 mentions "gainfully employed", O'Brien 2003 uses "open employment", Xiang 2007 "salaried employment" and Waghorn 2014 describes the temporary use of subsidies. However, we decided to include them based on types of jobs obtained and classifications between job types.

Secondary outcomes

Employment

1. Number of weeks in competitive employment

We extracted data (mean and SD) about weeks in competitive employment from 22 studies. Some trials used days or months in employment. We calculated the number of weeks by dividing the number of days by 5 or by multiplying the number of months by 4.5. Fourteen studies (Bejerholm 2015; Bond 2015b; Bond 2007; Drake 1999b; Hoffmann 2012; Killackey 2008; Latimer 2006; McGurk 2007; McGurk 2009; Michon 2014; Mueser 2004; Oshima 2014; Twamley 2012a; Wong 2008) reported the mean duration of employment for all participants. Seven RCTs presented data about only those participants who obtained competitive employment (Bond 1986; Bond 1995; Burns 2007; Penk 2010; Schonebaum 2006; Tsang 2001; Tsang 2010; Walker 1969). Bond 2007; Drebing 2005 and Michon 2014 reported both. We could not use data about maintaining employment from ten RCTs (Au 2015; Burns 2015; Craig 2014; Drebing 2007; Gervey 1994; Gold 2006; Howard 2010; Lecomte 2014; McFarlane 2000) because they did not report means with SDs, and we could not calculate them from the presented data. Viering 2015 did not report separate results for participants who were unemployed at baseline, and Drake 2013 only reported the results of paid employment, without identifying those who had been employed competitively.

2. Number of days to first competitive employment

Fifteen RCTs reported data for this outcome (Bejerholm 2015; Bond 1995; Bond 2007; Burns 2015; Drake 1999b; Drake 2013; Gold 2006; Hoffmann 2012; Howard 2010; Latimer 2006; Michon 2014; Mueser 2004; Penk 2010; Twamley 2012a; Wong 2008). We were able to use data from nine studies for the analyses of days to first competitive employment for those participants who became competitively employed (Bond 2007; Gold 2006; Hoffmann 2012; Latimer 2006; Michon 2014; Mueser 2004; Penk 2010; Twamley 2012a; Wong 2008).

3. Percentage of participants who obtained non‐competitive employment

For this outcome, 23 RCTs presented data (Becker 1967; Bejerholm 2015; Blankertz 1996; Bond 1986; Bond 2007; Bond 2015b; Chandler 1996; Craig 2014; Drake 1999b; Drake 2013; Gold 2006; Hoffmann 2012; Lecomte 2014; Lehman 2002; McFarlane 1996; McFarlane 2000; Michon 2014; Mueser 2004; O'Brien 2003; Oshima 2014; Twamley 2012a; Walker 1969; Wong 2008). However, in nine trials transitional employment was also an intervention or control condition (Becker 1967; Bond 1986; Bond 2007; Drake 1999b; Gold 2006; Hoffmann 2012; Mueser 2004; Oshima 2014; Walker 1969). Types of non‐competitive employment were transitional employment, volunteer jobs and other forms of paid employment.

Clinical outcomes

1. Quality of life

Twelve RCTs presented quality‐of‐life data. The Quality of Life Interview (QOLI) (Lehman 1988) was used by four studies (Bond 2007, Chandler 1996; Drake 1999b; Drake 2013) and the Lancashire Quality of Life Profile (QOLP) (Olliver 1997) was used in one study (Burns 2007). These are both five‐point scales. One study (Hoffmann 2012) used the Wisconsin Quality of Life index (W‐QLI) (Becker 1999), four studies (Bejerholm 2015; Burns 2015; Howard 2010; Michon 2014) presented data from the Manchester Short Asessment of Quality of Life (MANSA) (Priebe 1999), a seven‐point scale, and two studies (Au 2015; Tsang 2010) used the Personal Wellbeing Index (PWI) (Lau 2005). We could not use data from Bejerholm 2015 because this RCT showed medians instead of means. Au 2015 and Burns 2015 were not included in the analyses because the intervention and control group are classified as the same main intervention.

2. Mental health

Seven RCTs (Bond 2007; Burns 2007; Hoffmann 2012; McFarlane 1996; McGurk 2007; McGurk 2009; Xiang 2007) presented end score data from the Positive and Negative Syndrome Scale (PANSS) (Kay 1987). McFarlane 1996 reported only results for the whole group, but we excluded this study from our analyses because we could not classify the intervention. Au 2015; Burns 2015; Drake 1996; Drake 1999b; Eack 2009 and Howard 2010 reported data from the Brief Psychiatric Rating Scale (BPRS) (Overall 1962). Eack 2009 used the Wing Negative Symptom Scale (Wing 1961), Raskin Depression Scale (Raskin 1969), Covi Anxiety Scale (Lipman 1982) and Patient Subjective Response Questionnaire (Hogarty 1995). Other scales used were the Hamilton Depression and Anxiety Scale (HADS) (Zigmond 1983) by Burns 2007 and Burns 2015, the Short Form Health Survey (Ware 1996) by Drake 2013 and the Mental Health Inventory (MHI) (Berwick 1991) by Michon 2014.

Adverse events

1. Dropouts

All except seven trials reported number of dropouts (Beard 1963; Drebing 2007; Gervey 1994; Lecomte 2014; McFarlane 2000; Tsang 2001; ; Walker 1969). Beard 1963 reported an approximate percentage with differences between outcomes and without exact numbers. Therefore, we could not use these data.

2. Hospital admissions

Twenty‐one RCTs described hospitalisations, yet only 14 RCTs (Beard 1963; Becker 1967; Bond 1986; Bond 2015b; Burns 2007; Burns 2015; Chandler 1996; Dincin 1982; Howard 2010; Gold 2006; Michon 2014; O'Brien 2003; Walker 1969; Xiang 2007) provided data that we could use (number of participants who were admitted during follow‐up). Five studies (Drake 1999b; Drake 2013; Hoffmann 2012; McFarlane 2000; McGurk 2007) did not report the numbers or percentage of participants in both intervention and control group who were admitted during follow‐up. Two RCTs (Bond 1986; Burns 2015) could not be included in this analysis because we classified the intervention and control groups to be in the same intervention main group.

See Table 2 for a summary of the descriptive details of included studies and Table 3 for a list of the comparisons studied in included studies.

Excluded studies

We have outlined our reasons for excluding 61 RCTs in the Characteristics of excluded studies. The main reasons for excluding studies were lack of competitive employment outcomes, wrong study design and including a large proportion of participants who were already employed at baseline.

Studies awaiting classification

Six studies are awaiting classification (see Characteristics of studies awaiting classification: Bejerholm 2017; Glynn 2017; McGurk 2015; McGurk 2016; Kane 2015; Schneider 2016).

Ongoing studies

Eight studies are ongoing (see Ongoing studies: Bell 2015; Bitter 2015; Christensen 2015; Granholm 2014; Harris 2015; Melau 2011; Nordt 2012; Sveinsdottir 2014).

Risk of bias in included studies

Our judgements regarding the risk of bias in the individual studies are illustrated in Figure 4 and Figure 5. Details are described in Characteristics of included studies. Overall, we considered the risk of bias in the majority of the included studies as high because of the lack of blinding, incomplete outcome data and insufficient information about allocation concealment. One RCT Beard 1963 scored high on all items except for other potential bias. We judged the risk of bias of an individual study as high if four or more items of the 'Risk of bias' assessments were unclear or high, as moderate with two to three items unclear or high. Otherwise we judged the risk of bias of an individual study as low.

Risk of bias summary: review authors' judgements about each risk of bias item for each included study.

Risk of bias graph: review authors' judgements about each risk of bias item presented as percentages across all included studies.

Allocation

All trials stated that they randomly assigned participants, however, only 27 RCTs (56%) described an acceptable allocation procedure (low risk). Twenty‐RCTs (42%) did not provide sufficient information to assess sequence generation (unclear risk) and one RCT (Beard 1963) used a method of "rotation at application", based on the time and date of the subject's application, that we classified as high risk of bias (2%). Of 48 RCTs, 18 trials (38%) reported an adequate method to ensure allocation concealment (low risk), 28 trials (58%) did not provide enough information to give a judgement (unclear risk), Beard 1963 described an unconcealed procedure (high risk) and Craig 2014 recruited participants after randomisation of the teams (high risk).

Blinding

Due to the type of interventions (psychosocial), blinding of participants and personnel is difficult. However, one study (2%) (Tsang 2001) did describe blinding of participants low risk). Ten RCTs (21%) (Bejerholm 2015; Burns 2007; Burns 2015; Craig 2014; Hoffmann 2012; Howard 2010; Killackey 2008; Killackey 2014; Latimer 2006; Waghorn 2014) stated that the participants and personnel were not blinded to intervention allocation (high risk). We also classified the risk of bias as high in 35 RCTs (73%) because the participants and personnel could identify the intervention by the contents of the programme. Two trials (McGurk 2009; Waghorn 2014) (4%) did not report sufficient information about blinding of outcome measurements.

Eight RCTs (17%) ensured blinding of outcome measurements (Au 2015; Bejerholm 2015; Killackey 2014; McFarlane 1996; Michon 2014; Tsang 2001; Tsang 2010; ; Xiang 2007) (low risk). Three RCTs (6%) (Penk 2010; McGurk 2009; Waghorn 2014) did not provide enough information about blinding of participants and personnel (unclear risk). Sixteen RCTs (33%)(Bond 2007; Bond 2015b; Burns 2007; Burns 2015; Drake 1999b; Drebing 2007; Eack 2009; Gold 2006; Hoffmann 2012; Howard 2010; Killackey 2008; Latimer 2006; Mueser 2004; O'Brien 2003; Oshima 2014; Twamley 2012b) described the lack of blinding of outcome measurements. The remaining 21 studies (44%) did not report any details about blinding, but we classified them as high risk of bias based on their description of their methods of data collection.

Incomplete outcome data

The number of dropouts ranged from none (Oshima 2014) to 37% (Dincin 1982). Out of 48 studies, we considered 18 studies (37%) as having a low risk of bias due to incomplete outcome data (low number lost to follow‐up, balanced numbers with similar reasons, intention‐to‐treat analyses). Twenty‐seven studies (56%) were at high risk, and three trials (Blankertz 1996; McFarlane 2000; Tsang 2001) did not provide enough information (unclear risk).

Selective reporting

Almost all RCTs (83%) reported all pre‐specified outcomes. We judged four trials at high risk of bias (Beard 1963; Blankertz 1996; Gervey 1994; Lehman 2002). Three of these (Beard 1963; Blankertz 1996; Gervey 1994) had only published preliminary results, several years previously, and we were unable to identify a complete publication through our search; and Lehman 2002 did not report all listed outcomes. We did not have enough information for four other studies to make a judgement: in Killackey 2014; Lecomte 2014 and Nuechterlein 2012 only preliminary data were available, however, because these data were published recently, we expect a full report in the (near) future; Viering 2015 did not describe some outcomes but it is possible that they will be presented in a separate article.

Other potential sources of bias

We found the majority of RCTs (69%) not to have a risk of other biases. We classified the risk as high in four studies (8%). In Gold 2006, project redesign may have compromised study validity. The participants in Waghorn 2014 were allowed to switch from control to intervention group after six months. The participants in McFarlane 1996, in both the intervention and control groups, were treated by the same team and the study authors describe possible clinician bias. Killackey 2008 mentions a significant baseline difference in marital status with more chance to obtain a job if married. We found some evidence for other biases in eleven RCTs (23%), but the information was not enough to classify them as high risk of bias. Five trials (Becker 1967; Gervey 1994; Nuechterlein 2012; Tsang 2001; Walker 1969) did not report the funding source. Blankertz 1996 describes a difference between the intervention and control group in severity of disease or functioning, and working history. The first author of Bond 1986 was also the director of the institute. The three studies by Drake 2013; Hoffmann 2012 and Michon 2014 were funded by the social security agency that provided both income for the participants and, in the case of Hoffmann 2012 and Drake 2013, also the participants. In one study (Schonebaum 2006) two separate articles were published in the same journal with different authors and analyses. Neither set of authors knew of the others’ efforts according to the journal.

Five trials (Gold 2006; Lehman 2002; McFarlane 2000; Mueser 2004; Schonebaum 2006) included in this review were part of the Employment Intervention Demonstration programme (EIDP), a multisite collaboration among eight research demonstration sites in the USA with separate study designs. See Characteristics of excluded studies for other publications related to this programme. However, not all sites published data and this may be a source of publication bias. Also, some trials (Drebing 2005; Gervey 1994; Killackey 2014; Lecomte 2014; McGurk 2007; McGurk 2009; Oshima 2014) had small sample sizes (fewer than 50 participants), which could have led to false positive or negative outcomes.

Effects of interventions

See: Table 1

See Table 1.

Primary outcome

Percentage or number of participants who obtained competitive employment

Pairwise meta‐analyses/direct comparisons

For the primary outcome we differentiated between long‐term and short‐term follow‐up results and between intervention main groups and subgroups (see Description of the intervention).

Analysis 1.1 and Analysis 1.2 show the results of meta‐analyses of direct comparisons of the interventions. We used a random‐effects model because of the high level of heterogeneity (I² greater than 50%). In total 2425 out of 7523 participants (32%) obtained competitive employment during study follow‐up.

1.1 — Comparison 1 Any intervention to improve obtaining employment compared to another intervention in adults with severe mental illness, Outcome 1 Obtaining competitive employment, short‐term follow‐up (≤ 1 year).

1.2 — Comparison 1 Any intervention to improve obtaining employment compared to another intervention in adults with severe mental illness, Outcome 2 Obtaining competitive employment, long‐term follow‐up (> 1 year).

Short‐term follow‐up

In 18 RCTs, N = 2291, the follow‐up duration was one year or less. In Analysis 1.1 the pairwise meta‐analyses are reported for the comparisons supported employment, prevocational training and transitional employment versus psychiatric care only, supported employment and prevocational training versus transitional employment, and augmented supported employment versus supported employment.

Three comparisons showed significant results. The results were in favour of prevocational training compared to psychiatric care only (RR 8.96, 95% CI 1.77 to 45.51), meaning that participants who received prevocational training were more likely to obtain competitive employment than those who were only treated with psychiatric care. Furthermore, supported employment was more effective than transitional employment (RR 3.49, 95% CI 1.77 to 6.89) and prevocational training (RR 2.52, 95% CI 1.21 to 5.24).

Long‐term follow‐up

We included 22 RCTs (N = 5233) with a follow‐up duration of more than one year in this meta‐analysis. In Analysis 1.2 we report the pairwise meta‐analyses for the comparisons augmented supported employment, supported employment and prevocational training compared to psychiatric care only, augmented supported employment and supported employment compared to transitional employment, augmented supported employment and supported employment versus prevocational training and augmented supported employment versus supported employment only. Augmented supported employment (RR 4.32, 95% CI 1.49 to 12.48), supported employment (RR 1.51, 95% CI 1.36 to 1.68) and prevocational training (RR 2.19, 95% CI 1.07 to 4.46) were more effective than psychiatric care only. The results favoured augmented supported employment in the comparison with transitional employment (RR 2.45, 95% CI 1.69 to 3.55), PVT (RR 5.42, 95% CI 1.08 to 27.11) and supported employment (RR 1.94, 95% CI 1.03 to 3.65). Additionally, supported employment was more effective than transitional employment(RR 3.28, 95% CI 2.13 to 5.04) and prevocational training (RR 2.31, 95% CI 1.85 to 2.89). See the forest plot for all comparisons.

Network meta‐analyses

Table 1 provides overall estimates of the effect of the intervention in obtaining competitive employment, and the quality of evidence. We performed network meta‐analyses of the long‐term follow‐up results, for both the intervention main and subgroups.

Intervention main groups (long‐term follow‐up)

The direct comparisons are demonstrated in Figure 2. We included 22 RCTs (N = 5233) in this network meta‐analysis. Supported employment compared to prevocational training was investigated most (9 RCTs, N = 784). We did not include Schonebaum 2006 in the main group network meta‐analysis because we classified both the intervention and control condition as augmented supported employment.

Figure 6 and Table 3 show the estimates of benefit of each intervention against each other. The SUCRA values and ranking of interventions are displayed in Table 4 and Figure 7. Augmented supported employment was the most effective intervention versus psychiatric care only in obtaining competitive employment (RR 3.81, 95% CI 1.99 to 7.31, SUCRA 98.5, mean rank 1.1), followed by supported employment(RR 2.72, 95% CI 1.55 to 4.76; SUCRA 76.5, mean rank 1.9). Prevocational training (RR 1.26, 95% CI 0.73 to 2.19; SUCRA 40.3, mean rank 3.4) and transitional employment(RR 1.00, 95% CI 0.51 to 1.96; SUCRA 17.2, mean rank 4.3) compared to psychiatric care only did not show relative risk differences in competitive employment rate. Augmented supported employment and supported employment were both more effective than transitional employment and prevocational training. We did not find a risk difference between transitional employment and prevocational training, but prevocational training stands out in the SUCRA value and rank compared to transitional employment. In the comparison between augmented supported employment and supported employment the results slightly favoured augmented supported employment, but not significantly (RR 1.40, 95% CI 0.92 to 2.14). The SUCRA value and mean rank were higher for augmented supported employment.

Network meta‐analysis estimates of intervention benefit.

CI: confidence interval; **Psych care**: psychiatric care only; **PVT**: prevocational employment; RR: risk ratio; SE: supported employment; **SE+**: augmented supported employment; TE: transitional employment;

Plots of the surface under the cumulative ranking curves (SUCRAs) for the interventions included in the network (long‐term follow‐up).

**PVT**: prevocational training; **Psych care**: psychiatric care only; SE: supported employment; **SE+**: augmented supported employment; TE: transitional employment

See Table 4 for effectiveness of interventions on obtaining competitive employment (long‐ term follow‐up).

3. Effectiveness of interventions on obtaining competitive employment (long‐ term follow‐up).

SE+
1.40 (0.92 to 2.14)	SE
3.02 (1.88 to 4.87)	2.16 (1.59 to 2.93)	PVT
3.79 (2.34 to 6.14)	2.71 (1.80 to 4.06)	1.26 (0.77 to 2.05)	TE
3.81 (1.99 to 7.31)	2.72 (1.55 to 4.76)	1.26 (0.73 to 2.19)	1.00 (0.51 to 1.96)	Psych care
Network meta‐analysis estimates of intervention effect (RR with 95% CI). The column intervention is compared with the row intervention. RR > 1 favours the column intervention.

Open in a new tab

Psych care: psychiatric care only  PVT: prevocational employment;  SE: supported employment  SE+: augmented supported employment  TE: transitional employment

See Table 5 for relative ranking of estimated probabilities (long‐term follow‐up).

4. Relative ranking of estimated probabilities (long‐term follow‐up).

Intervention	SUCRA	mean rank
SE+	98.5	1.1
SE	76.5	1.9
PVT	40.3	3.4
TE	17.2	4.3
Psychiatric care only	17.5	4.3

Open in a new tab

SUCRA = surface under the cumulative ranking curve  PVT: prevocational training  SE: supported employment  SE+: augmented supported employment  TE: transitional employment

Intervention subgroups (long‐term follow‐up)

Figure 3 shows the network of the intervention subgroups. We included 24 RCTs (N = 5656) in this network meta‐analysis. Most trials compared high‐fidelity IPS to job‐related skills training (7 RCTs, N = 624). Several direct comparisons were missing. See Table 3 for the comparison classifications of the intervention subgroups per study. For this analysis we combined the RCTs with supported employment combined with cognitive training or social skills training (SE plus symptom‐related skills training). See Table 6 and Table 7 for the effectiveness of the interventions classified in subgroups. Figure 8 shows the forest plot and Figure 9 presents the SUCRA values and ranking. Supported employment plus symptom‐related skills training showed the best results (RR compared to psychiatric care only 3.61, 95% CI 1.03 to 12.63, SUCRA 80.3, mean rank 3.2). The other augmented supported employment interventions were also more effective than psychiatric care only and were ranked above IPS, Clubhouse, sheltered workshops and job‐related skills training. The results of the comparison of cognitive training and psychiatric care only favoured cognitive training, but not significantly (RR 2.88, 95% CI 0.60 to 13.87). However, the SUCRA value was high (78.4%) and this intervention was ranked second (mean rank 3.4). Social skills training and IPS were more effective than psychiatric care only, Clubhouse, sheltered workshops and job‐related skills training. We did not find risk differences between high‐fidelity IPS and low‐fidelity IPS (RR 1.20, 95% CI 0.62 to 2.35). Job‐related skills training, sheltered workshops and Clubhouse were not more effective than psychiatric care only.

5. Effectiveness of intervention subgroups on obtaining competitive employment (long‐term follow‐up).

SE + job
0.82 (0.09 to 7.17)	SE + symp
1.03 (0.10 to 11.00)	1.26 (0.22 to 7.04)	SE + TE
0.89 (0.11 to 7.18)	1.08 (0.29 to 4.04)	0.86 (0.28 to 2.63)	SE + ACT
1.73 (0.23 to 12.82)	2.10 (0.93 to 4.76)	1.67 (0.37 to 7.63)	1.94 (0.69 to 5.44)	hf IPS
2.08 (0.25 to 17.21)	2.53 (1.14 to 5.63)	2.02 (0.38 to 10.58)	2.34 (0.68 to 7.99)	1.20 (0.62 to 2.35)	lf IPS
5.46 (0.63 to 47.60)	6.64 (2.09 to 21.16)	5.29 (0.94 to 29.68)	6.14 (1.64 to 22.89)	3.16 (1.39 to 7.18)	2.63 (0.91 to 7.58)	CH
2.38 (0.30 to 18.92)	2.89 (0.91 to 9.16)	2.30 (0.58 to 9.09)	2.67 (1.19 to 5.96)	1.37 (0.61 to 3.09)	1.14 (0.40 to 3.27)	0.44 (0.14 to 1.38)	SWS
4.62 (0.59 to 35.99)	5.62 (2.44 to 12.95)	4.47 (0.92 to 21.76)	5.19 (1.69 to 15.95)	2.67 (1.70 to 4.20)	2.22 (1.17 to 4.23)	0.85 (0.33 to 2.16)	1.94 (0.77 to 4.89)	Job
1.63 (0.44 to 6.08)	1.98 (0.36 to 11.06)	1.58 (0.22 to 11.27)	1.83 (0.36 to 9.25)	0.94 (0.21 to 4.27)	0.78 (0.15 to 4.09)	0.30 (0.05 to 1.66)	0.69 (0.14 to 3.41)	0.35 (0.07 to 1.70)	SST
0.68 (0.06 to 8.42)	0.83 (0.09 to 7.35)	0.66 (0.06 to 7.14)	0.77 (0.09 to 6.28)	0.39 (0.05 to 2.98)	0.33 (0.04 to 2.76)	0.12 (0.01 to 1.11)	0.29 (0.04 to 2.32)	0.15 (0.02 to 1.17)	0.42 (0.05 to 3.56)	CT
2.97 (0.51 to 17.40)	3.61 (1.03 to 12.63)	2.88 (0.60 to 13.87)	3.34 (1.10 to 10.13)	1.72 (0.67 to 4.42)	1.43 (0.45 to 4.55)	0.54 (0.16 to 1.90)	1.25 (0.42 to 3.71)	0.64 (0.23 to 1.83)	1.82 (0.56 to 5.93)	4.35 (0.73 to 25.98)	Psych care
Network meta‐analysis estimates of intervention effect (RR, 95% CI). The column intervention is compared with the row intervention. RR > 1 favours the column intervention.

Open in a new tab

CH: Clubhouse  CT: cognitive training  hf IPS: high‐fidelity Individual Placement and Support  Job : job‐related skills training  lf IPS: low‐fidelity Individual Placement and Support  Psych care: psychiatric care only  SE + ACT: supported employment + assertive community treatment  SE + job: supported employment + job‐related skills training  SE + symp: supported employment + symptom‐related skills training  SE + TE: supported employment + transitional employment  SST: social skills training  SWS: sheltered workshops

6. Relative ranking of estimated probabilities of intervention subgroups (long‐term follow‐up).

Intervention	SUCRA	mean rank
SE + symp	80.3	3.2
CT	78.4	3.4
SE + ACT	77.8	3.4
SE + TE	69	4.4
SE + job	68.4	4.5
Hf IPS	51.9	6.3
SST	51.8	6.3
lf IPS	42.4	7.3
SWS	35.8	8.1
Psychcare	25.8	9.2
Job	10.6	10.8
CH	7.9	11.1

Open in a new tab

SUCRA = surface under the cumulative ranking curve  CH: Clubhouse  CT: cognitive training  hf IPS: high‐fidelity Individual Placement and Support  Job : job‐related skills training  lf IPS: low‐fidelity Individual Placement and Support  Psych care: psychiatric care only  SE + ACT: supported employment + assertive community treatment  SE + job: supported employment + job‐related skills training  SE + symp: supported employment + symptom‐related skills training  SE + TE: supported employment + transitional employment  SST: social skills training  SWS: sheltered workshops

Effectiveness of intervention subgroups in obtaining competitive employment (long‐term follow‐up).

CH: Clubhouse; CI: confidence interval; CT: cognitive training; **hf IPS**: high‐fidelity Individual Placement and Support; **job training**: job‐related skills training; **lf IPS**: low‐fidelity Individual Placement and Support;**Psych care only**: psychiatric care only; RR: risk ratio; **SE + ACT**: supported employment + assertive community treatment; **SE + job**: supported employment + job‐related skills training; **SE + symp**: supported employment + symptom‐related skills training; **SE + TE**: supported employment + transitional employment; **SST**: social skills training; **SWS**: sheltered workshops

Plots of the surface under the cumulative ranking curves (SUCRAs) for the interventions subgroups included in the network (long‐term follow‐up)

CH: Clubhouse; CT: cognitive training; **hf IPS**: high‐fidelity Individual Placement and Support; **job:** job‐related skills training; **lf IPS**: low‐fidelity Individual Placement and Support;**Psych care**: psychiatric care only; **SE + ACT**: supported employment + assertive community treatment; **SE + job**: supported employment + job‐related skills training; **SE + symp**: supported employment + symptom‐related skills training; **SE + TE**: supported employment + transitional employment; **SST**: social skills training; **SWS**: sheltered workshops

See Table 6 for effectiveness of intervention subgroups on obtaining competitive employment (long‐term follow‐up).

See Table 7 for relative ranking of estimated probabilities of intervention subgroups (long‐term follow‐up).

Secondary outcomes

Employment

1. Number of weeks in competitive employment

Four short term follow‐up trials (Bond 1986; Walker 1969; Penk 2010; Drebing 2005) and five long term follow‐up trials ( Bond 2007; Burns 2007; Tsang 2010; Schonebaum 2006; Michon 2014) showed data about job duration for only those participants who obtained a competitive job. The competitively employed participants worked on average for a duration of 13.26 weeks (SD 10.72, N = 109s) in studies with short‐term follow‐up and 32.74 weeks (SD 18.74, N = 443 participants) in long‐term follow‐up trials.

See meta‐analyses of weeks worked in competitive employment in Analysis 1.1 and Analysis 1.2. We used a random‐effects model due to the high level of heterogeneity (I² >50%). We could not perform a network meta‐analysis for maintaining competitive employment because of inconsistency, heterogeneity in type of data and the very small number of studies.

Short‐term follow‐up

We obtained data about job maintenance, the number of weeks in competitive employment, from eight RCTs (N = 478) (Bond 1995; Bond 2015b; Killackey 2008; Latimer 2006; Oshima 2014; Penk 2010; Twamley 2012a; Walker 1969). See Analysis 2.1 for all comparisons and mean differences (MD) in weeks worked.

2.1 — Comparison 2 Any intervention to improve maintaining employment compared to another intervention in adults with severe mental illness, Outcome 1 Weeks in competitive employment, short‐term follow‐up (≤ 1 year).

Supported employment was more effective than psychiatric care only, transitional employment and prevocational training. The biggest difference was between supported employment and prevocational training, favouring supported employment (MD 6.89 weeks, 95% CI 1.26 to 12.52). For the comparisons augmented supported employment versus supported employment the results favoured supported employment. However, this analysis contains only one study (Bond 1995) with augmented supported employment as the control condition.

Long‐term follow‐up

We included eleven RCTs, N = 1102, in the direct meta‐analysis (Bejerholm 2015; Bond 2007; Burns 2007; Drake 1999b; Hoffmann 2012; McGurk 2007; McGurk 2009; Michon 2014; Mueser 2004; Tsang 2010; Wong 2008). All meta‐analyses show results favouring supported employment and augmented supported employment, see Analysis 2.2 and the forest plot. Participants receiving supported employment worked more weeks than those receiving transitional employment (MD 17.36, 95% CI 11.53 to 23.18) or prevocational training (MD 11.56, 95% CI 5.99 to 17.13). Augmented supported employment was more effective than supported employment in maintaining employment (MD 10.09, 95% CI 0.32 to 19.85). In the comparison between augmented supported employment and prevocational training the results also favoured augmented supported employment (MD 22.79, 95% CI 15.96 to 29.62).

2.2 — Comparison 2 Any intervention to improve maintaining employment compared to another intervention in adults with severe mental illness, Outcome 2 Weeks in competitive employment, long‐term follow‐up (> 1 year).

2. Number of days to first competitive employment

Short‐term follow‐up

The pooled and separate data of the three individual studies with short‐term follow‐up (Latimer 2006; Penk 2010; Twamley 2012a) showed no difference between the interventions in time to first competitive employment, see Analysis 3.1.

3.1 — Comparison 3 Any intervention to improve obtaining employment compared to another intervention in adults with severe mental illness, Outcome 1 Days to first competitive employment, short‐term follow‐up (≤ 1 year).

Long‐term follow‐up

Six RCTs (Bond 2007; Gold 2006; Hoffmann 2012; Michon 2014; Mueser 2004; Wong 2008) with long‐term follow‐up provided data for this outcome. Participants in the augmented supported employment group and supported employment group obtained employment faster than those receiving transitional employment, see Analysis 3.2 and the forest plot.

3.2 — Comparison 3 Any intervention to improve obtaining employment compared to another intervention in adults with severe mental illness, Outcome 2 Days to first competitive employment, long‐term follow‐up (> 1 year).

3. Percentage of participants who obtained non‐competitive employment

We present the results of the meta‐analyses of obtaining non‐competitive employment in Analysis 4.1 and Analysis 4.2. We used a random‐effects model because of the high level of heterogeneity (I² >50%). For these analyses we included RCTs with transitional employment as both the intervention (Becker 1967; Penk 2010; Walker 1969) and control condition (Bond 2007; Drake 1999b; Gold 2006; Hoffmann 2012; Latimer 2006; McFarlane 2000; Mueser 2004; Oshima 2014), because in the majority of these trials fewer than half of the participants worked in non‐competitive employment (including transitional employment positions), so apparently the availability of placements did not guarantee obtaining jobs or working in these jobs.

4.1 — Comparison 4 Any intervention to improve obtaining employment compared to another intervention in adults with severe mental illness, Outcome 1 Obtaining non‐competitive employment, short‐term follow‐up (≤ 1 year).

4.2 — Comparison 4 Any intervention to improve obtaining employment compared to another intervention in adults with severe mental illness, Outcome 2 Obtaining non‐competitive employment, long‐term follow‐up (> 1 year).

Short‐term follow‐up

We included 11 RCTs (Becker 1967; Blankertz 1996; Bond 2015b; Craig 2014; Latimer 2006; Lecomte 2014; O'Brien 2003; Oshima 2014; Penk 2010; Twamley 2012a; Walker 1969) in this analysis. The results show no differences in the non‐competitive employment rate between the interventions, except for the results of the comparison transitional employment versus prevocational training. In this comparison the results favoured prevocational training (RR 3.82, 95% CI 2.24 to 6.53). However this analysis included only one trial (Penk 2010). See Analysis 4.1 for all comparisons.

Long‐term follow‐up

Non‐competitive employment data were reported in 13 RCTs with long‐term follow‐up. We included five comparisons. See Analysis 4.2. In one trial (Chandler 1996) augmented supported employment was more effective than psychiatric care only (RR 44.69, 95% CI 6.25 to 319.49). Two trials (Gold 2006; McFarlane 2000) compared augmented supported employment to transitional employment and found inconclusive results (RR 0.44, 95% CI 0.12 to 1.66). Transitional employment also showed better results in comparison to supported employment (Bond 1995; Drake 1999b; Hoffmann 2012; Mueser 2004) (RR 0.22, 95% CI 0.08 to 0.63). The large trial of Drake 2013 comparing supported employment to psychiatric care found no difference between these interventions.

Clinical outcomes

1. Quality of life

Long‐term follow‐up

Quality‐of‐life end‐score data were reported in 10 RCTs with long‐term follow‐up (Bejerholm 2015; Bond 2007; Burns 2007; Chandler 1996; Drake 1999b; Drake 2013; Hoffmann 2012; Howard 2010; Michon 2014; Tsang 2010). See Analysis 5.1 for all comparisons, three of which produced significant differences.

5.1 — Comparison 5 Any intervention to improve obtaining or maintaining employment compared to another intervention in adults with severe mental illness, Outcome 1 Quality of life, long‐term follow up (> 1 year).

Chandler 1996 found a significant difference in favour of psychiatric care only compared to augmented supported employment (SMD ‐0.79, 95% CI ‐1.05 to ‐0.55) when effectiveness was measured with the Quality of Life Instrument (QOLI, Lehman 1988). Drake 2013 also used the QOLI. Supported employment scored higher than psychiatric care only (SMD 0.14, 95% CI 0.06 to 0.23).

Tsang 2010 used the Personal Wellbeing Index (Lau 2005). Participants in the augmented supported employment group scored higher than those in the prevocational training group (SMD 0.41, 95% CI 0.06 to 0.77). On a five‐point scale, these differences are not clinically relevant.

2. Mental health

Long‐term follow‐up

Seven RCTs (Bond 2007; Burns 2007; Drake 1999b; Drake 2013; Eack 2009; Michon 2014; Xiang 2007) produced end‐score data with mental health instruments. All of them had a follow‐up duration of more than one year. Two comparisons showed significant results (see Analysis 6.1). In Xiang 2007 the participants in the prevocational training group scored significantly lower on all scales of the PANNS (Kay 1987): positive symptoms (SMD ‐2.48, 95% CI ‐3.95 to ‐1.01) negative symptoms (SMD ‐1.61, 95% CI ‐2.99 to ‐0.23) and general psychopathology (SMD ‐1.86, 95% CI ‐3.09 to ‐0.63). Drake 2013 used the SFHS (Ware 1996). In this big trial the authors found a significant difference in end scores in favour of supported employment compared to psychiatric care only (SMD 2.88, 95% CI 1.78 to 3.98).

6.1 — Comparison 6 Any intervention to improve obtaining or maintaining employment compared to another intervention in adults with severe mental illness, Outcome 1 Mental health long‐term follow‐up (> 1 year).

Adverse events

1. Dropouts

Almost all included studies (N = 41) reported the number of study dropouts. We found no risk differences in the comparisons in Analysis 7.1 (short‐term follow‐up) or Analysis 7.2 (long‐term follow‐up). The median dropout percentage of the short‐term follow‐up trials (N = 2439) was 15%, ranging from none (Oshima 2014) to 37% (Dincin 1982). The percentages were similar in the long‐term trials (N = 5423): median 14%, ranging from 1% (Wong 2008) to 43% (Michon 2014).

7.1 — Comparison 7 Any intervention to improve obtaining or maintaining employment compared to another intervention in adults with severe mental illness, Outcome 1 Dropouts, short‐term follow‐up (≤ 1 year).

7.2 — Comparison 7 Any intervention to improve obtaining or maintaining employment compared to another intervention in adults with severe mental illness, Outcome 2 Dropouts, long‐term follow‐up (> 1 year).

2. Hospital admissions

Short‐term follow‐up

We included seven RCTs (Beard 1963; Becker 1967; Bond 2015b; O'Brien 2003; Walker 1969; Beard 1963; Dincin 1982) in the meta‐analyses of hospital admissions (see Analysis 8.1). The results did not favour any specific intervention.

8.1 — Comparison 8 Any intervention to improve obtaining or maintaining employment compared to another intervention in adults with severe mental illness, Outcome 1 Hospital admissions, short‐term follow‐up (≤ 1 year).

Long‐term follow‐up

Five RCTs (Chandler 1996; Gold 2006; Burns 2007; Howard 2010; Michon 2014) reported the number of hospital admissions during long‐term follow‐up. Analysis 8.2 shows all comparisons. One of the comparisons found prevocational training leading to less hospital admissions compared to psychiatric care only (RR 0.26, CI 0.11‐0.65, based on one study: Xiang 2007).

8.2 — Comparison 8 Any intervention to improve obtaining or maintaining employment compared to another intervention in adults with severe mental illness, Outcome 2 Hospital admissions, long‐term follow‐up (> 1 year).

Assessment of heterogeneity and inconsistency within the network

Transitivity assumption

The plausibility of the transitivity assumption requires judgement to decide whether differences in the distribution of the effect modifiers across trials are large enough to make network meta‐analyses invalid. We evaluated the transitivity by considering the differences between studies. As is commonly observed in social or non‐pharmacotherapeutic intervention studies, there was variation in participants and interventions between the trials. For example, two studies (Drake 2013; Viering 2015) mainly included participants with major affective disorders compared to psychotic disorders in the other studies. However, we still considered the transitivity assumption not to be violated.

Heterogeneity and inconsistency

We performed an assessment of heterogeneity (between‐study variance) and inconsistency (difference between direct and indirect evidence) within the network meta‐analyses. We evaluated the loop‐specific inconsistency and the inconsistency of the total network for both the intervention main group and subgroup network meta‐analysis of long‐term follow‐up studies. We could not perform the network meta‐analysis of the short‐term follow‐up studies due to the high possibility of inconsistency (loop‐specific risk odds ratios (ROR) up to 13, network inconsistency P value = 0.02).

Heterogeneity

The heterogeneity within the comparisons showed no evidence of heterogeneity except for the comparison augmented supported employment versus prevocational training (Tau² greater than 50% quantile). One loop in the network (in which this comparison was included), also showed moderate heterogeneity (Tau² greater than 50% quantile).

Loop‐specific inconsistency

The comparisons in the network of the intervention main groups formed five closed loops of evidence. Figure 10 presents loop‐specific ROR, estimated as a difference between direct and indirect treatment estimates in loops of the network. For three loops the ROR was larger than 2, meaning that the direct estimates could be twice as large as the indirect estimate or the opposite. However, the 95% CIs did not cross the ROR = 1 line in only one loop (psychiatric care only‐supported employment‐prevocational training). For two out of ten comparisons (supported employment versus psychiatric care only and prevocational training versus supported employment) the predictive interval potentially changed the interpretation of the findings, since it crossed the RR = 1 line when the CIs for the means did not. See Figure 11 for the ROR of the loops in the intervention subgroup analyses. Three loops showed a ROR greater than 2, however the CIs did not cross the ROR = 1 line. For the subgroup interventions network, 10 out of 66 comparisons had predictive intervals crossing the RR = 1 line, these were comparisons including supported employment plus symptom‐related skills training, supported employment plus job‐related skills training and job‐related skills training only compared to psychiatric care only, Clubhouse and sheltered workshops, and supported employment plus symptom‐related skills training versus high‐fidelity IPS, and job‐related skills training versus low‐fidelity IPS.

Inconsistency plots for long‐term follow‐up and loop‐specific heterogeneity estimates.

**PVT**: prevocational training; **Psych care**: psychiatric care only; **ROR**: risk odds ratio; SE: supported employment; **SE+**: augmented supported employment; TE: transitional employment

Loop‐specific inconsistency in intervention subgroup network (long‐term follow‐up). **hf IPS**: high‐fidelity Individual Placement and Support; **job**: job‐related skills training; **lf IPS**: low‐fidelity Individual Placement and Support;**Psych care**: psychiatric care only; SE: supported employment;TE: transitional employment; **SWS**: sheltered workshops

Network inconsistency

When evaluating the inconsistency in the networks as a whole, there was an indication of global inconsistency within the network of the main group interventions. In the global test for inconsistency (design‐by‐treatment model) in the network of main group interventions the P value was < 0.001 and for the network of subgroup interventions P = 0.145.

Grading the quality of evidence

We graded the evidence for the network estimates for all the comparisons (see Table 1) according to the strategy by Salanti 2014, differentiating between the confidence in the specific pairwise effect estimates and the estimated treatment ranking. Salanti 2014 developed a specific method for network meta‐analysis, which is integrated with the GRADE approach (GRADE Working Group 2013). We considered the following domains: study limitations, indirectness, inconsistency, imprecision of effect estimates, and risk of publication bias, and graded the quality of evidence for each outcome as high, moderate, low or very low. High means that we are confident that the true effect is close to the estimate. Moderate confidence means that the true effect is likely to be close to the estimate, low means that there may be a difference and very low means that there is very little confidence in the estimate.

Study limitations

We assessed the study limitations for each network estimate by determining what percentage of direct comparisons contributed to the network meta‐analysis treatment effect, integrated with the 'Risk of bias' assessments of the direct and indirect evidence in the comparisons (Figure 12). The percentage contribution of each comparison to the network and comparison estimates are displayed in the contribution matrix (Figure 13). Two comparisons contained only indirect evidence, meaning that we did not include studies comparing these interventions directly and all evidence came from other comparisons. The comparisons supported employment versus prevocational training contributed predominately to the entire network. Each comparison contributes to all mixed and indirect estimates. We assessed the risk of bias of each comparison. Thus, each percentage of contribution to an estimate or to the entire network corresponds with a risk of bias. In Figure 12 we show the percentage of levels of risk of bias of direct and indirect evidence per comparison. The risk of bias of the direct evidence of a comparison is based on the risk of bias of the individual studies included in the comparison. The indirect evidence is based on the risk of bias of the included direct comparisons within the network. The percentage of contribution of the studies per comparison is shown in Figure 13. We judged the risk of bias of an individual study as high if four or more items of the 'Risk of bias' assessments were unclear or high, as moderate with two to three items unclear or high. Otherwise we judged the overall risk of bias for an individual study to be low. See Risk of bias in included studies and Figure 5 for the assessment of individual studies. If we included more than one study in a comparison, we based the judgement on the mean risk of bias. All studies within the comparisons of the network had an overall moderate to high risk of bias. We downgraded the level of evidence for the comparison (augmented supported employment versus transitional employment) with two levels because most information came from studies with high risk of bias. We downgraded the level of evidence for all other comparisons with one level as most information came from studies with a moderate risk of bias.

Contribution matrix: Percentage contribution of each direct estimate to the NMA estimates. **PVT**: prevocational training; **Psych care**: psychiatric care only; SE: supported employment; **SE+**: augmented supported employment; TE: transitional employment

Indirectness

We evaluated differences in populations, interventions and outcomes between trials, and compared the distribution of effect modifiers across comparisons. We only included trials with long‐term follow‐up in the network meta‐analysis. We decided that downgrading for indirectness was not necessary.

Inconsistency

The assessment of heterogeneity and inconsistency is described above. We downgraded the level of evidence for the comparisons augmented supported employment versus prevocational training by one level because the level of heterogeneity (Tau²) exceeded the 50% quantile. We downgraded the level of evidence for the comparisons supported employment versus psychiatric care only, supported employment versus prevocational training, and prevocational training versus psychiatric care only because the ROR of this loop was 3.156 (95% CI 1.46 to 6.84). Additionally, two of these comparisons (supported employment versus psychiatric care only and supported employment versus prevocational training) showed a predictive interval that potentially changed the interpretation of the findings since it crossed the RR = 1 line when the confidence interval for the mean did not.

Imprecision

We assessed imprecision by focusing on width of the confidence interval. We downgraded the level of evidence for four comparisons (augmented supported employment versus supported employment, prevocational training versus psychiatric care only, transitional employment versus psychiatric care only and prevocational training versus transitional employment) by one level because the confidence intervals included values that could favour either of the compared interventions.

Publication bias

We performed searches based on a comprehensive search strategy, including a search for unpublished studies. For a network meta‐analysis a comparison‐adjusted funnel plot can be helpful to detect possible small study effects (Chaimani 2013). Figure 14 shows the comparison‐adjusted funnel plot for the network of interventions for obtaining competitive employment (long‐term follow‐up). We placed the interventions in increasing order of their focus on competitive job search, because we think that small studies with negative results of the more job‐focused interventions (supported employment or augmented supported employment) could be less reported. The zero line corresponds with no difference between the study‐specific effect sizes within a comparison compared to the pooled effect size of a comparison. The zero line means ratio of RR = 1. The missing small studies on the right side of the zero line suggests that small studies tend to exaggerate the effectiveness of treatments with more focus on competitive job search. Therefore we downgraded our confidence in the network and for the comparison supported employment versus prevocational training.

Comparison‐adjusted funnel plot for the network of intervention for obtaining competitive employment for people with severe mental illness (long‐term follow‐up). The black line represents the null hypothesis that the study‐specific effect sizes do not differ from the respective comparison‐specific pooled effect estimates. The blue line is the regression line. The interventions are ordered based on their focus on competitive job search. Missing small studies on the right side of the zero line (means ratio of RR > 1) suggests that small studies tend to exaggerate the effectiveness of interventions with more focus on competitive job search. **PVT**: prevocational training; **Psych care**: psychiatric care only; RR: risk ratio; SE: supported employment; **SE+**: augmented supported employment; TE: transitional employment

Quality of evidence of ranking

We downgraded the level of evidence for study limitations in the ranking because most information was derived from studies with moderate risk of bias. Downgrading for indirectness was not necessary for the direct comparisons, so we did not need to downgrade our confidence for the overall ranking. In the network we found evidence for possible global inconsistency (P = 0.001), therefore downgrading was necessary. The ranking is rather precise as can be seen in the certainty of intervention ranking (see Table 3). We downgraded the ranking one level further because we detected evidence of publication bias (see above).

See Table 8 for a summary of our confidence in effect estimates and ranking of interventions.

7. Summary of our confidence in effect estimates and ranking of interventions.

Comparison	evidence	confidence	reasons for downgrading
SE + vs psych care	mixed	moderate	study limitations^a
SE vs psych care	indirect	low	study limitations^a;inconsistency^b,c
PVT vs psych care	mixed	very low	study limitations^a; inconsistency^b;imprecision^d
TE vs psych care	indirect	low	study limitations^a; imprecision^d
SE + vs TE	mixed	low	study limitations^e
SE vs TE	mixed	moderate	study limitations^a
PVT vs TE	indirect	low	study limitations^a; imprecision^d
SE + vs PVT	mixed	low	study limitations^a inconsistency^f
SE vs PVT	mixed	very low	study limitations^a; inconsistency^b,c; publication bias^g
SE + vs SE	mixed	low	study limitations^a; imprecision^d
Ranking		very low	study limitations^a; inconsistency^h; publication bias^g

Open in a new tab

^a Dominated by evidence at high or moderate risk of bias.  ^b Predictive interval for intervention effect includes effect that would have different interpretations.

^c Moderate level of heterogeneity.  ^d Confidence intervals include values favouring either intervention.

^e Dominated by evidence at high risk of bias.

^f Loop inconsistency ROR 3.156 (95% CI 1.46 to 6.84).

^g Evidence for small study effects.

^h Evidence for inconsistency in the network (P = 0.001).

psych care: psychiatric care only  SE: supported employment  SE +: augmented supported employment  PVT: prevocational training  TE: transitional employment

Discussion

Summary of main results

This Cochrane systematic review about vocational rehabilitation interventions for adults with severe mental illness included 48 RCTs involving 8743 participants. Of these, we used 22 RCTs (5233 participants) in the network meta‐analysis for long‐term follow‐up. All trials provided data about the primary outcome, percentage or number of participants who obtained competitive employment. The length of follow‐up ranged from three months up to five years. The studies included in the network meta‐analysis had a follow‐up duration of more than one year. The results for short‐term follow up were in line with the long‐term follow up.

See Table 1 for an overview of the network meta‐analysis results for all comparisons.

Direct comparison meta‐analysis of obtaining competitive employment

In both the short‐term and long‐term follow‐up meta‐analyses, prevocational training was more effective than psychiatric care only. The results of the long‐term follow‐up meta‐analysis favoured augmented supported employment, supported employment and prevocational training compared to psychiatric care only. Supported employment was more effective than transitional employment and prevocational training in both meta‐analyses. Augmented supported employment produced better results compared to transitional employment, prevocational training and supported employment in the long‐term follow‐up meta‐analysis.

Network meta‐analysis of obtaining competitive employment

Based on moderate‐quality evidence, augmented supported employment was the most effective intervention when compared with psychiatric care only in obtaining competitive employment. Based on very low‐quality evidence the next most effective intervention in obtaining competitive employment was supported employment. There was no difference between prevocational training and transitional employment compared to psychiatric care only in terms of competitive employment rate, but prevocational training stands out in the SUCRA value and rank (low‐ to very low‐quality evidence). In the comparison between augmented supported employment and supported employment the results slightly favoured augmented supported employment, but not significantly. The SUCRA value and mean rank were higher for augmented supported employment. The results of the network meta‐analysis of the intervention subgroups favoured augmented supported employment interventions, but also cognitive training. However, supported employment plus symptom‐related skills training produced the best results.

Maintaining competitive employment in direct comparison meta‐analysis

In the long‐term follow‐up meta‐analysis, augmented supported employment was more effective than prevocational training and supported employment in maintaining competitive employment. Participants receiving supported employment worked more weeks than those receiving transitional employment or prevocational training. The results of the short‐term follow‐up meta‐analysis favoured supported employment compared to psychiatric care only, transitional employment, prevocational training and augmented supported employment.

Other secondary outcomes and adverse events in direct comparison meta‐analysis

Employment

Participants in the augmented supported employment and supported employment groups obtained competitive employment faster compared to transitional employment and prevocational training. Transitional employment was more effective than prevocational training and supported employment in obtaining non‐competitive employment. Augmented supported employment was more effective compared to psychiatric care only.

Clinical outcomes

The comparisons for quality of life showed a difference in effect depending on the scale for quality of life. One comparison favoured prevocational training compared to augmented supported employment, and two comparisons were more positive for psychiatric care only compared to supported employment and augmented supported employment. However, all these results were based on one study per analysis and the MDs were very minimal. The comparisons showed no difference in mental health.

Adverse events

The comparisons showed no difference in number of drop outs or the number of hospital admissions, except for one study with results favouring prevocational training compared to psychiatric care only.

Overall completeness and applicability of evidence

Completeness

The majority of RCTs (N = 30) included supported employment as intervention or control condition. They compared supported employment to all the other interventions, but mostly to prevocational training (N = 11). Thirteen studies included augmented supported employment interventions, of which six compared augmented supported employment to supported employment. Included studies had evaluated all intervention subgroups that we had prespecified, except for social enterprises. In the network meta‐analysis with long‐term follow‐up data, supported employment versus prevocational training was the most common direct comparison. The follow‐up duration of most trials (N = 27) was longer than one year, some even up to five years. The majority of comparisons for long‐term follow‐up contained direct evidence, except for the comparisons transitional employment versus prevocational training, transitional employment versus psychiatric care only and supported employment versus psychiatric care only. All included trials reported data about our primary outcome (obtaining competitive employment). We refrained from performing a network meta‐analysis for the short‐term follow‐up trials (obtaining competitive employment) because of the high level of inconsistency despite subgroup analysis.

Applicability

Most RCTs (N = 30/48) included in this Cochrane review were conducted in North America. We did not find any trials from low‐ or middle‐income countries. The results of this Cochrane review could be less applicable for these countries because their mental health care systems are less developed or community‐based, or both, and are not integrated with vocational rehabilitation programmes. In general, we think that the participants included in this Cochrane review are comparable to people with severe mental illness in community psychiatric care. The participants were mainly diagnosed with psychotic disorders followed by mood disorders, two thirds were men, their mean age was 36 years, and almost all participants were unemployed at baseline. Remarkably, most trials did not mention substance abuse or excluded participants with alcohol or drug use problems. Therefore, the results could be less applicable to mental health care programmes for people who have problems with substance use. Four trials exclusively included young adults with first episode psychosis (Craig 2014; Killackey 2008; Killackey 2014; Nuechterlein 2012). One study primarily focused on participants with criminal justice involvement (Bond 2015b). The participants of 44 out of 48 trials were fully unemployed at baseline, and the majority of them had participated in some form of employment in the past. Interest in (competitive) employment was an eligibility criterion in 34 out of 48 studies, particularly for supported employment studies because this intervention requires motivation to obtain a competitive job.

Supported employment interventions used the individual placement and support model and their good fidelity scores mean they were implemented well. The applicability of the interventions can be considered good because in western countries mental health care teams and vocational rehabilitation services are very common and they often provide these type of programmes. However, it is important to keep in mind that each country has its own social security and mental health care system that can influence the implementation, accessibility and effectiveness of the interventions. For example, two included studies required having a disability benefit for enrolment in the study (Drake 2013; Viering 2015).

Quality of the evidence

We graded the evidence for each separate comparison in the network and the overall ranking of the network of interventions for obtaining competitive employment (long‐term follow‐up). For a detailed description see section 'Grading the quality of evidence'. We downgraded our confidence in the evidence of all comparisons because of study limitations. We classified the majority of studies contributing to the comparisons in the network as moderate or high risk of bias, generally due to lack of blinding and incomplete outcome data. Therefore, we also downgraded the overall ranking correspondingly. We downgraded four out of 10 comparisons (augmented supported employment versus supported employment, transitional employment versus psychiatric care only, prevocational training versus psychiatric care only and prevocational training versus transitional employment) for imprecision. Additionally, it was necessary to downgrade three comparisons for inconsistency: augmented supported employment versus prevocational training because of heterogeneity and prevocational training versus supported employment and supported employment versus psychiatric care only because the predictive interval potentially changed the results. We downgraded two comparisons and the overall ranking in the network meta‐analysis because we found evidence for possible global inconsistency in the design‐by‐treatment model and the loop‐specific approach.

Unfortunately, we could not perform network meta‐analysis for the short‐term follow‐up studies because the loop‐specific inconsistency was too high. In the long‐term follow‐up meta‐analysis we identified some variation in participants, interventions and outcomes between trials. However, this variation is common in psychosocial intervention studies and we still consider the transitivity assumption to hold because of our specific inclusion criteria and distinction between follow‐up duration. Therefore we deemed it not necessary to downgrade the quality of evidence for indirectness. We detected evidence of publication bias in the network meta‐analysis and in the comparison supported employment versus prevocational training. It is possible that small studies with less positive results have not been published. Therefore downgrading the quality of evidence for publication bias was necessary. In conclusion, our confidence in the quality of evidence of the intervention effects varied from moderate to very low, mainly because of study limitations but also due to imprecision and inconsistency. We judged the overall ranking as very low because of study limitations, inconsistency, and evidence of publication bias.

Potential biases in the review process

We conducted systematic searches based on a comprehensive search strategy, including a search for systematic reviews, RCTs and also for unpublished studies. The comparison‐adjusted funnel plot shows evidence of small study effects. We included studies only when participants had been diagnosed with a severe mental illness with persisting disabilities. Most included studies included participants who had mainly psychotic disorders, but some focused on other diagnoses, as did the two largest trials (O'Brien 2003; Drake 2013).

Furthermore, the classification of interventions was complicated. Several interventions had components of more than one intervention category, especially in the prevocational training group. It cannot be ruled out that we misclassified an intervention. Similar interventions may still differ in intensity, duration and professionals involved.

Another limitation of our Cochrane review is the varying definitions of obtaining competitive employment. Some trials required a period of time being in competitive employment or a minimum number of hours worked a week, or both, before they counted having obtained a successful result. The definition used ranged from a minimum requirement of five days' Hoffmann 2012 to three months' (Xiang 2007) consecutive working and from five hours' (Oshima 2014) to 20 hours' (Drebing 2007; Tsang 2010) working a week. In the majority of the trials the type of jobs counted as competitive employment was well described, in agreement with our definition. We also identified four studies with less clear definitions. Beard 1963 mentions "gainfully employed", O'Brien 2003 uses "open employment", Xiang 2007 "salaried employment" and Waghorn 2014 describes the temporary use of subsidies. However, we decided to include them based on types of jobs obtained and classifications between job types.

Unfortunately, data about maintaining employment were much more diverse, making it impossible to perform a network meta‐analysis. We found the biggest difference in type of data. For the long‐term follow‐up, six trials reported data for all participants whereas five trials reported job duration data of only those participants who had worked in a competitive job. We performed a direct comparison meta‐analysis but we had to downgrade the quality of evidence because of this heterogeneity.

Outcome reporting bias

There is a possibility of outcome reporting bias because we only included studies that reported competitive employment outcomes. Furthermore, a reasonable part of the studies did not report quality of life or mental health outcomes. Therefore it is possible that contradictory results were not published or included.

Agreements and disagreements with other studies or reviews

As far as we know, this is the first systematic review including a network meta‐analysis about vocational rehabilitation interventions for adults with severe mental illness. Several systematic reviews, with and without meta‐analysis have been published (Almerie 2015; Arbesman 2011; Bond 2015a; Bouvet 2014; Crowther 2001; Dieterich 2010; Heffernan 2011; Kinoshita 2013; Twamley 2003). The results of their meta‐analyses and narrative descriptions are consistent with our results showing the benefit of supported employment in obtaining and maintaining competitive employment compared to other approaches. The low quality of the evidence was also described previously by Kinoshita 2013. Some reviews found differences that we did not identify. Bouvet 2014 considered the Clubhouse model (transitional employment) more effective than other psychosocial rehabilitation programmes (not supported employment), but this narrative systematic review also included non‐RCTs. Social skills training was not better than standard care or discussion groups in Almerie 2015. Finally, Dieterich 2010 found a trend towards intensive case management (psychiatric care) being better than standard care.

Authors' conclusions

Implications for practice.

We found very low‐ to moderate‐quality evidence that supported employment and augmented supported employment were more effective in obtaining and maintaining competitive employment compared to prevocational training, transitional employment or psychiatric care only, without increasing dropouts or hospital admissions. We found low‐quality evidence that augmented supported employment may be slightly more effective than supported employment alone, based on the direct comparison meta‐analysis and SUCRA value in the network meta‐analysis, but the risk difference in effect was no longer significant in the network meta‐analysis. However, supported employment plus symptom‐related skills training was significantly more effective than the other interventions in the subgroup network meta‐analysis. We found no evidence of a considerable negative or positive effect on quality of life or mental health for any vocational rehabilitation intervention. This is remarkable because an important reason for stimulating people to participate in work is to enhance their quality of life. On the other hand we also found no evidence for adverse events such as dropping out of the programme or hospitalisation.

Implications for research.

The quality of randomised controlled trials (RCTs) has improved over the years. However, most research has focused on supported employment and on augmented supported employment, which limits a broad perspective on all available vocational intervention methods or other innovative interventions. Furthermore, future studies should try to minimise risk of bias by trying to decrease the dropout rate and to blind participants and outcome assessors.

The majority of RCTs have, up to now, focused on people with psychotic disorders, with the exception of a few trials also focusing on people with affective disorders. These other patient categories, also including mental illness with substance abuse should be further explored to get a full understanding of the potential of vocational rehabilitation programmes for different populations. We also recommend studies exploring the influence of motivation on the effectiveness of vocational rehabilitation programmes, particularly for supported employment and augmented supported employment, as studies evaluating these interventions report a zero exclusion criterion.

Cost‐effectiveness trials are needed to investigate whether the possible extra effect is worth the extra money. To illustrate, trials focusing on the effectiveness of social enterprises are lacking but these interventions are widely used. Therefore inclusion of these interventions in research should be considered. Convincing evidence of cost‐effectiveness of an intervention should stimulate implementation. Also, augmented supported employment is a very heterogeneous group of programmes, combining supported employment with other interventions. Future research should focus more on the difference in effectiveness of the various types of the additional interventions compared to supported employment only or to each other, especially when combined with social skills training or cognitive training. These specific prevocational interventions yield better results than other prevocational training, transitional employment and psychiatric care only interventions. Therefore it would be interesting to further investigate these training programmes. We recommend the use of programme fidelity audits to enhance the comparability of interventions from different studies.

We recommend using a clear definition and operationalisation of competitive employment as an outcome in evaluation studies of vocational rehabilitation programmes. And more studies are needed to evaluate the effectiveness of maintaining competitive employment. In the end, we believe that being able to keep a job will have the largest effect on an individual's quality of life compared to just finding a job.

Acknowledgements

We thank Kaisa Neuvonen, Cochrane Work Information Specialist, for developing the search strategy. We also thank Jos Verbeek, co‐ordinating Editor, Cochrane Work, for his useful comments and contribution to the data‐analysis. We thank Denise Mitchell and Jani Ruotsalainen for copy editing the text.

Appendices

Appendix 1. MEDLINE search strategy for systematic reviews

#1 (Severe) mental disorders

"Mental Disorders"[Mesh] OR "mentally ill persons"[mesh] OR "mentally ill"[tw] OR "mental disorder"[tw] OR "mental disorders"[tw] OR "mental disease"[tw] OR "mental diseases"[tw] OR "mental illness"[tw] OR "mental illnesses"[tw] OR schizophreni*[tw] OR "Psychotic Disorders"[Mesh] OR psychosis[tw] OR psychotic[tw] OR "bipolar disorder"[tw] OR "bipolar disorders"[tw] OR "major depressive disorder"[tw] OR "major depressive disorders"[tw] OR "personality disorder"[tw] OR "personality disorders"[tw] OR "anxiety disorder"[tw] OR "anxiety disorders"[tw] OR "post‐traumatic stress disorder"[tw] OR "post‐traumatic stress disorders"[tw] OR PTSD[tw] OR autis*[tw]

#2 Return to work, work (dis)ability, sick leave and presenteeism, vocational rehabilitation

“Return to Work”[Mesh] OR “return to work”[tw] OR return‐to‐work[tw] OR “work participation”[tw] OR “work performance”[tw] OR “performance at work”[tw] OR work function*[tw] OR “employment status”[tw] OR “work status”[tw] OR “work ability”[tw] OR workability[tw] OR ”work capacity“[tw] OR ”work activity“[tw] OR ”work disability“[tw] OR ”work rehabilitation“[ tw] OR ”work status“[tw] OR ”work retention“[tw] OR employability[tw] OR employable[tw] OR employee*[tw] OR employment[MeSH Terms] OR employment OR unemployment[MeSH Terms] OR unemployment[tw] OR unemployed[tw] OR retirement[tw] OR (early[tw]) AND (retirement[Mesh]) OR “early retirement”[tw] OR “presenteeism”[tw] OR “Absenteeism”[Mesh] OR “absenteeism”[tw] OR “disability absence”[tw] OR “sickness absence”[tw] OR sick leave[MeSH Terms] OR sick leave[tw] OR “work absenteeism”[tw] OR “work absence”[tw] OR “work day loss”[tw] OR “work time loss”[tw] OR “work productivity”[tw] OR “occupational health”[tw] OR “occupational health”[Mesh] OR occupational health services[MeSH Terms] OR “Disability evaluation”[Mesh] OR “disability evaluation”[tw] OR ”disability management“[tw] OR “disability prevention”[tw] OR “work capacity evaluation”[Mesh]OR “work capacity evaluation”[tw] OR “work disability”[tiab] OR “work incapacity”[tiab] OR “work incapability”[tiab] OR “work inhibition”[tw] OR “working incapacity”[tw] OR (“disabled persons”[Mesh] AND work[MeSH Terms] OR occupations[MeSH Terms] OR occupation* OR vocational*) OR “sick leave“[Mesh] OR “medical leave”[tw] OR “sick leave”[tw] OR (“disability”[tw]) AND (“pension”[Mesh]) OR “disability pension”[tw] OR ”Rehabilitation, Vocational“[Mesh] OR “vocational rehabilitation”[tw] OR “cognitive training”[tw] OR “clubhouse model”[tw] OR ”Sheltered Workshops“[Mesh] OR “social skills training”[tw] OR ”Employment, Supported“[Mesh] OR “supported employment”[tw] OR “individual placement or support”[tw] OR “individual placement”[tw] OR “individual support”[tw]

#3 (Systematic) Reviews

"Review" [Publication Type] OR "systematic review"[tw] OR ”intervention review”[tw] OR “review” [tw] OR "Meta‐Analysis" [Publication Type] OR "Meta‐Analysis" [tw] OR "Meta Analysis" [tw] OR "Meta‐Analyses" [tw] OR "Meta Analyses"[tw]

#1 AND #2 AND #3

#4 animals [mh] NOT humans [mh]

#4 NOT #5

Appendix 2. MEDLINE search strategy for RCTs

#1 (Severe) mental disorders

#2 Return to work, work (dis)ability, sick leave and presenteeism, vocational rehabilitation

#3 RCTs

("randomized controlled trial"[pt] OR "controlled clinical trial"[pt] OR "Randomized Controlled Trials as Topic"[mh] OR "Controlled Clinical Trials as Topic"[mh] OR "Random Allocation"[mh] OR "Double‐Blind Method"[mh] OR "Single‐Blind Method"[mh] OR "clinical trial"[pt] OR "Clinical Trials as Topic"[mh] OR "clinical trial"[tw] OR ((singl*[tw] OR doubl*[tw] OR trebl*[tw] OR tripl*[tw]) AND (mask*[tw] OR blind*[tw])) OR "latin square"[tw] OR Placebos[mh] OR placebo*[tw] OR random*[tw] OR Epidemiological Studies[mh] OR "Research Design"[mh:noexp] OR "Comparative Study"[mh] OR "evaluation studies"[pt] OR "Evaluation Studies As Topic"[mh] OR "Follow‐Up Studies"[mh] OR "Prospective Studies"[mh] OR "Cross‐Over Studies"[mh] OR control*[tw] OR prospectiv*[tw] OR volunteer*[tw]) NOT (Animals[mh] NOT Humans[mh])

#1 AND #2 AND #3

#4 animals [mh] NOT humans [mh]

#4 NOT #5

Appendix 3. CENTRAL search strategy for systematic reviews and RCTs

#1 MeSH descriptor: [Mental Disorders] explode all trees

#2 "mental illness":ti,ab,kw (Word variations have been searched)

#3 "schizophrenia":ti,ab,kw (Word variations have been searched)

#4 MeSH descriptor: [Psychotic Disorders] explode all trees

#5 "psychosis":ti,ab,kw (Word variations have been searched)

#6 "psychotic disorder":ti,ab,kw (Word variations have been searched)

#7 "bipolar disorder":ti,ab,kw (Word variations have been searched)

#8 "major depressive disorder":ti,ab,kw (Word variations have searched)

#9 "personality disorder":ti,ab,kw (Word variations have been searched)

#10 "anxiety disorder":ti,ab,kw (Word variations have been searched)

#11 "post‐traumatic stress disorder":ti,ab,kw (Word variations have s)

#12 "PTSD":ti,ab,kw (Word variations have been searched)

#13 "autistic disorder":ti,ab,kw (Word variations have been searched)

#14 "autism":ti,ab,kw (Word variations have been searched)

#15 #1 or #2 or #3 or #4 or #5 or #6 or #7 or #8 or #9 or #10 or #11  or #12 or #13 or #14

#16 MeSH descriptor: [Return to Work] explode all trees

#17 "return to work":ti,ab,kw (Word variations have been searched)

#18 "work participation":ti,ab,kw (Word variations have been searched)

#19 "work performance":ti,ab,kw (Word variations have been searched)

#20 "performance at work":ti,ab,kw (Word variations have been searched)

#21 "work function":ti,ab,kw (Word variations have been searched)

#22 "employment status":ti,ab,kw (Word variations have been searched)

#23 "work status":ti,ab,kw (Word variations have been searched)

#24 #16 or #17 or #18 or #19 or #20 or #21 or #22 or #23

#25 "work ability":ti,ab,kw (Word variations have been searched)

#26 "work capacity":ti,ab,kw (Word variations have been searched)

#27 "work activity":ti,ab,kw (Word variations have been searched)

#28 "work disability":ti,ab,kw (Word variations have been searched)

#29 "work rehabilitation":ti,ab,kw (Word variations have been searched)

#30 "work status":ti,ab,kw (Word variations have been searched)

#31 "work retention":ti,ab,kw (Word variations have been searched)

#32 "employability":ti,ab,kw (Word variations have been searched)

#33 "employable":ti,ab,kw (Word variations have been searched)

#34 employe*:ti,ab,kw (Word variations have been searched)

#35 MeSH descriptor: [Employment] explode all trees

#36 "employment":ti,ab,kw (Word variations have been searched)

#37 MeSH descriptor: [Unemployment] explode all trees

#38 "unemployment":ti,ab,kw (Word variations have been searched)

#39 "unemployed":ti,ab,kw (Word variations have been searched)

#40 "retirement":ti,ab,kw (Word variations have been searched)

#41 MeSH descriptor: [Retirement] explode all trees

#42 early and #41

#43 "early retirement":ti,ab,kw (Word variations have been searched)

#44 #25 or #26 or #27 or #28 or #29 or #30 or #31 or #32 or #33 or #34  or #35 or #36 or #37 or #38 or #39 or #40 or #41 or #42 or #43

#45 "presenteeism":ti,ab,kw (Word variations have been searched)

#46 MeSH descriptor: [Absenteeism] explode all trees

#47 "absenteeism":ti,ab,kw (Word variations have been searched)

#48 "disability absence":ti,ab,kw (Word variations have been searched)

#49 "sickness absence":ti,ab,kw (Word variations have been searched)

#50 MeSH descriptor: [Sick Leave] explode all trees

#51 "sick leave":ti,ab,kw (Word variations have been searched)

#52 "work absenteeism":ti,ab,kw (Word variations have been searched)

#53 "work absence":ti,ab,kw (Word variations have been searched)

#54 "work day loss":ti,ab,kw (Word variations have been searched)

#55 "work time loss":ti,ab,kw (Word variations have been searched)

#56 "work productivity":ti,ab,kw (Word variations have been searched)

#57 MeSH descriptor: [Occupational Health] explode all trees

#58 "occupational health":ti,ab,kw (Word variations have been searched)

#59 MeSH descriptor: [Occupational Health Services] explode all trees

#60 #45 or #46 or #47 or #48 or #49 or #50 or #51 or #52 or #53 or #54  or #55 or #56 or #57 or #58 or #59

#61 MeSH descriptor: [Disability Evaluation] explode all trees

#62 "disability evaluation":ti,ab,kw (Word variations have been searched)

#63 "disability management":ti,ab,kw (Word variations have been searched)

#64 "disability prevention":ti,ab,kw (Word variations have been searched)

#65 MeSH descriptor: [Work Capacity Evaluation] explode all trees

#66 "work capacity evaluation":ti,ab,kw (Word variations have been s)

#67 "work disability":ti,ab,kw (Word variations have been searched)

#68 "work incapacity":ti,ab,kw (Word variations have been searched)

#69 "work incapability":ti,ab,kw (Word variations have been searched)

#70 "work inhibition":ti,ab,kw (Word variations have been searched)

#71 "working incapacity":ti,ab,kw (Word variations have been searched)

#72 MeSH descriptor: [Disabled Persons] explode all trees

#73 MeSH descriptor: [Work] explode all trees

#74 MeSH descriptor: [Occupations] explode all trees

#75 occupation*:ti,ab,kw (Word variations have been searched)

#76 vocational*:ti,ab,kw (Word variations have been searched)

#77 #72 and (#73 or #74 or #75 or #76)

#78 MeSH descriptor: [Sick Leave] explode all trees

#79 "sick leave":ti,ab,kw (Word variations have been searched)

#80 "medical leave":ti,ab,kw (Word variations have been searched)

#81 "disability":ti,ab,kw (Word variations have been searched)

#82 MeSH descriptor: [Pensions] explode all trees

#83 #81 and #82

#84 "disability pension":ti,ab,kw (Word variations have been searched)

#85 #61 or #62 or #63 or #64 or #65 or #66 or #67 or #68 or #69 or #70 or  #71 or #77 or #78 or #79 or #80 or #83 or #84

#86 MeSH descriptor: [Rehabilitation, Vocational] explode all trees

#87 "vocational rehabilitation":ti,ab,kw (Word variations have been s)

#88 "cognitive training":ti,ab,kw (Word variations have been searched)

#89 "clubhouse model":ti,ab,kw (Word variations have been searched)

#90 MeSH descriptor: [Sheltered Workshops] explode all trees

#91 "social skills training":ti,ab,kw (Word variations have been searched)

#92 MeSH descriptor: [Employment, Supported] explode all trees

#93 "supported employment":ti,ab,kw (Word variations have been s)

#94 "individual placement or support":ti,ab,kw (Word variations have)

#95 "individual placement":ti,ab,kw (Word variations have been searched)

#96 "individual support":ti,ab,kw (Word variations have been searched)

#97 #86 or #87 or #88 or #89 or #90 or #91 or #92 or #93 or #94 or #95 or #96

#98 #24 or #44 or #60 or #77 or #97

#99 #15 and #98

#100 #99 in Cochrane reviews

#101 #99 in other reviews

#102 #99 in trials

Appendix 4. Embase search strategy for systematic reviews

'mental disease'/exp OR 'mental disease' OR 'mental illness'/de OR 'mental illness' OR 'schizoaffective disorder'/de OR 'schizoaffective disorder' OR 'schizophrenia'/exp OR 'schizophrenia' OR 'psychotic disorders'/exp OR 'psychotic disorders' OR 'psychosis'/exp OR psychosis OR 'psychotic disorder' OR 'bipolar disorder'/exp OR 'bipolar disorder' OR 'major depressive disorder'/exp OR 'major depressive disorder' OR 'major depression'/exp OR 'major depression' OR 'personality disorder'/exp OR 'personality disorder' OR 'anxiety disorder'/exp OR 'anxiety disorder' OR 'post‐traumatic stress disorder'/exp OR 'post‐traumatic stress disorder' OR 'ptsd'/exp OR 'ptsd' OR 'autistic disorder'/exp OR 'autistic disorder' OR 'autism'/exp OR 'autism'

'return to work'/exp OR 'return to work' OR 'work participation' OR 'work performance' OR 'job performance' OR performance NEAR/5 work OR 'work function' OR 'employment status' OR 'work status'

'work ability' OR 'work capacity' OR workability OR 'work activity' OR 'work disability'/exp OR 'work disability' OR work NEAR/5 rehabilitation OR 'work status' OR 'work retention' OR employability'/exp OR employability OR employable OR employee OR 'employment'/exp OR 'employment' OR 'unemployment'/exp OR 'unemployment' OR 'unemployed'/exp OR unemployed OR 'retirement'/exp OR 'retirement' OR (early AND ('retirement'/exp OR retirement)

'presenteeism' OR 'absenteeism' OR 'disability absence'/exp OR 'disability absence' OR 'sickness absence' OR 'sick leave'/exp OR 'sick leave' OR 'medical leave'/exp OR 'medical leave' OR 'work absenteeism' OR 'work absence' OR 'work day loss' OR 'work time loss' OR 'work productivity' OR 'occupational health' OR 'occupational health'/exp OR 'occupational health service'/exp

disability NEAR/5 evaluation OR 'disability management' OR 'disability prevention' OR 'work capacity'/exp OR 'work capacity evaluation' OR 'work capacity' NEAR/5 evaluation OR 'work disability':ab,ti OR 'work incapacity':ab,ti OR 'work incapability':ab,ti OR 'work inhibition' OR 'working incapacity' OR ('disabled person'/exp AND ('work'/exp OR 'occupation'/exp OR occupation* OR vocational* OR 'medical leave'/exp OR 'medical leave' OR 'sick leave' OR ('disability' AND 'pension'/exp) OR 'disability pension'

'vocational rehabilitation'/exp OR 'vocational rehabilitation' OR 'cognitive training' OR 'clubhouse model' OR 'sheltered workshop'/exp OR 'social skills training' OR 'supported employment' OR 'individual placement support' OR 'individual placement' OR 'individual support'

#2 OR #3 OR #4 OR #5 OR #6

#8 #1 AND #7

'review'/exp OR 'systematic review': ti,ab OR 'meta analysis'/exp OR 'metaanalysis':ti,ab OR 'metaanalyses':ti,ab OR 'meta analysis':ti,ab OR 'meta analyses':ti,ab

#10

#9 OR #10

#11

#8 AND #11

#12

#11 AND [embase]/lim NOT [medline]/lim

Appendix 5. Embase search strategy for RCTs

'return to work'/exp OR 'return to work' OR 'work participation' OR 'work performance' OR 'job performance' OR performance NEAR/5 work OR 'work function' OR 'employment status' OR 'work status'

#2 OR #3 OR #4 OR #5 OR #6

#8 #1 AND #7

random* OR factorial* OR crossover* OR cross NEXT/1 over* OR placebo* OR doubl* NEAR/1 blind* OR singl* NEAR/2 blind* OR assign* OR allocat* OR volunteer*

#10

'crossover procedure'/exp OR 'double blind procedure'/exp OR 'single blind procedure'/exp OR 'randomized controlled trial'/exp OR 'randomized controlled trial (topic)'/exp

#11

#9 OR #10

#12

#8 AND #11

#13

#12 AND [embase]/lim NOT [medline]/lim

Appendix 6. CINAHL search strategy for systematic reviews

#1 Severe mental illness

(MH "Mental Disorders+") OR (TX mental* ill*) OR (MH "Mental Disorders, Chronic+") OR (MH "Schizophrenia+") OR (TX schizophrenia) OR (MH "Psychotic Disorders+") OR (TX psychosis) OR (TX psychotic disorder) OR (MH "Schizoaffective Disorder+") OR (MH "Affective Disorders, Psychotic+") OR (MH "Bipolar Disorder+") OR (TX bipolar disorder) OR( TX personality disorder) OR (MH “Depression+”) OR (TX major depressive disorder) OR (MH Anxiety Disorders+)”OR (TX anxiety disorder) OR (MH "Stress Disorders, Post‐Traumatic+") OR (TX post‐traumatic stress disorder) OR (MH “autistic disorder+”) OR (TX autistic disorder) OR (TX autism)

#2 RTW

(MH "Job Re‐Entry+") OR (TX return to work) OR (MH "Social Participation+") OR (MH “Job Performance+”) OR (MH “Employment+”) OR (TX employ*) OR (TX work ability) OR (TX workability) OR (TX work capacity) OR (TX work* disab*) OR (MH "Unemployment") OR (TX unemploy*) OR (MH "Presenteeism") OR (MH "Absenteeism") OR (MH "Sick Leave") OR (TX absenteeism) OR (TX sickness absence) OR (TX sick leave) OR (TX work productivity) OR (TX work* incapacity) OR (TX disability pension) OR (MH "Employee, Disabled+") OR (MH "Insurance Benefits") OR (MH "Insurance, Disability+") OR (MH "Insurance, Unemployment") OR (TX disability benefit) OR (MH "Occupational Health+") OR (MH "Occupational Health Services+") OR (MH "Occupational Medicine") OR (TX occupation*) OR (MH "Disability Management") OR (MH "Disability Evaluation+") OR (TX disability evaluation) OR (TX disability management) OR (MH "Rehabilitation, Psychosocial+") OR (MH "Rehabilitation, Vocational+") OR (MH "Rehabilitation, Community‐Based") OR (TX psychosocial rehabilitation) OR (TX vocation*) OR (TX vocational rehabilitation) OR (TX supported employment) OR (MH "Employment, Supported") OR (MH "Employment of Disabled+") OR (MH "Sheltered Workshops") OR (TX sheltered employment) OR (MH "Rehabilitation, Cognitive") OR (MH "Cognitive Therapy") OR (MH "Social Skills Training") OR (MH "Vocational Education") OR (TX cognitive training) OR (TX cognitive therapy) OR (TX cognitive rehabilitation) OR (TX social skills) OR (MH "Social Skills") OR (TX individual placement and support)

#3 Systematic review

(MH “Systematic Review”) OR (MH “Meta Analysis”) or (TI (meta‐analy* OR metaanaly*)) OR (AB (meta‐analy* OR metaanaly*)) OR (PT systematic review) OR (TI systematic review OR AB systematic review)

#1 AND 2 AND 3#

Appendix 7. CINAHL search strategy for RCTs

#1 Severe mental illness

#2 RTW

#3 RCT

(MH "Randomized Controlled Trials") OR (MH "Clinical Trials+") OR (PT clinical trial) OR (PT randomized controlled trial) OR (TX clinical trial) OR (TX randomi* control* trial*) OR (MH "Random Assignment") OR (TX random* allocat*) OR (TX allocate* random*) OR (TX placebo*) OR (MH “Placebos”) OR (MH “Quantitative Studies”)

#1 AND 2 AND 3#

Appendix 8. PsycINFO search strategy for systematic reviews

# 1 severe mental illness

(SU.EXACT("Mental Disorders") OR "mental disorder" OR "mental illness" OR schizophrenia OR psychosis OR "psychotic disorder" OR "bipolar disorder" OR "major depressive disorder" OR "personality disorder" OR "anxiety disorder" OR "post‐traumatic stress disorder" OR "PTSD" OR "autistic disorder" OR autism)

# 2 RTW

(SU.EXACT("Return to Work") OR "return to work" OR return‐to‐work OR "work participation" OR "work performance" OR "performance at work" OR work function* OR "employment status" OR "work status") OR ("work ability" OR workability OR "work capacity" OR "work activity" OR "work disability" OR "work rehabilitation" OR "work status" OR "work retention" OR employability OR employable OR employe* OR employment OR SU.EXACT("Employment status") OR SU.EXACT("Unemployment") OR unemployment OR unemployed OR retirement OR "early retirement" OR early NEAR/1 retirement) OR (presenteeism OR SU.EXACT("Absenteeism") OR absenteeism OR "disability absence" OR "sickness absence" OR "sick leave" OR SU.EXACT("Employee Leave Benefits") OR "work absenteeism" OR "work absence" OR "work day loss" OR "work time loss" OR "work productivity" OR "occupational health" OR SU.EXACT("Occupational Health")) OR (SU.EXACT("Vocational Rehabilitation") OR "vocational rehabilitation" OR "cognitive training" OR "clubhouse model" OR SU.EXACT("Sheltered Workshops") OR "social skills training" OR SU.EXACT("Supported Employment") OR "supported employment" OR "individual placement" OR "individual support") OR ((SU.EXACT("Disability evaluation") OR "disability evaluation" OR "disability management" OR "disability prevention" OR ("work capacity" NEAR/5 evaluation) OR "work capacity evaluation" OR "work disability" OR "work incapacity" OR "work incapability" OR "work inhibition" OR "working incapacity") OR ("disability pension" OR (disability AND pension) OR (disability AND SU.EXACT("Employee Pension Plans"))))))

#3 systematic reviews

(TI "Systematic Review" OR "Meta Analysis" OR "Meta‐Analyses" OR "Meta Analyses") OR (AB "Systematic Review" OR "Meta Analysis" OR "Meta‐Analyses" OR "Meta Analyses")

#1 AND #2 AND #3

Appendix 9. PsycINFO search strategy for RCTs

# 1 severe mental illness

# 2 RTW

#3 RCTs

(SU.EXACT("Clinical Trials") OR random* OR factorial* OR crossover* OR placebo* OR assign* OR allocat* OR volunteer* OR (cross over*) OR (double blind*) OR (singl* blind*)

#1 AND #2 AND #3

Appendix 10. Cochrane 'Risk of bias' assessment tool

Random sequence generation Selection bias (biased allocation to interventions) due to inadequate generation of a randomised sequence
Criteria for a judgement of ‘low' risk of bias	The investigators describe a random component in the sequence generation process such as: referring to a random number table; using a computer random‐number generator; coin tossing; shuffling cards or envelopes; throwing dice; drawing of lots; minimisation. Minimisation may be implemented without a random element, and this is considered to be equivalent to being random.
Criteria for the judgement of ‘high' risk of bias	The investigators describe a non‐random component in the sequence generation process. Usually, the description would involve some systematic, non‐random approach, for example: sequence generated by odd or even date of birth; sequence generated by some rule based on date (or day) of admission; sequence generated by some rule based on hospital or clinic record number. Other non‐random approaches happen much less frequently than the systematic approaches mentioned above and tend to be obvious. They usually involve judgement or some method of non‐random categorisation of participants, for example: allocation by judgement of the clinician; allocation by preference of the participant; allocation based on the results of a laboratory test or a series of tests; allocation by availability of the intervention.
Criteria for the judgement of ‘unclear' risk of bias.	Insufficient information about the sequence generation process to permit judgement of ‘low' risk or ‘high' risk.
Allocation concealment Selection bias (biased allocation to interventions) due to inadequate concealment of allocations prior to assignment
Criteria for a judgement of ‘low' risk of bias	Participants and investigators enrolling participants could not foresee assignment because one of the following, or an equivalent method, was used to conceal allocation: central allocation (including telephone, web‐based and pharmacy‐controlled randomisation); sequentially numbered drug containers of identical appearance; sequentially numbered, opaque, sealed envelopes.
Criteria for the judgement of ‘high' risk of bias	Participants or investigators enrolling participants could possibly foresee assignments and thus introduce selection bias, such as allocation based on: using an open random allocation schedule (e.g. a list of random numbers); assignment envelopes were used without appropriate safeguards (e.g. if envelopes were unsealed or nonopaque or not sequentially numbered); alternation or rotation; date of birth; case record number; any other explicitly unconcealed procedure.
Criteria for the judgement of ‘unclear' risk of bias	Insufficient information to permit judgement of ‘low' risk or ‘high' risk. This is usually the case if the method of concealment is not described or not described in sufficient detail to allow a definite judgement – for example if the use of assignment envelopes is described, but it remains unclear whether envelopes were sequentially numbered, opaque and sealed.
Blinding of participants and personnel Performance bias due to knowledge of the allocated interventions by participants and personnel during the study
Criteria for a judgement of ‘low' risk of bias	Any one of the following: no blinding or incomplete blinding, but the review authors judge that the outcome is not likely to be influenced by lack of blinding; blinding of participants and key study personnel ensured, and unlikely that the blinding could have been broken.
Criteria for the judgement of ‘high' risk of bias	Any one of the following: no blinding or incomplete blinding, and the outcome is likely to be influenced by lack of blinding; blinding of key study participants and personnel attempted, but likely that the blinding could have been broken, and the outcome is likely to be influenced by lack of blinding.
Criteria for the judgement of ‘unclear' risk of bias	Any one of the following: insufficient information to permit judgement of ‘low' risk or ‘high' risk; the study did not address this outcome.
Blinding of outcome assessment Detection bias due to knowledge of the allocated interventions by outcome assessors
Criteria for a judgement of ‘low' risk of bias	Any one of the following: no blinding of outcome assessment, but the review authors judge that the outcome measurement is not likely to be influenced by lack of blinding; blinding of outcome assessment ensured, and unlikely that the blinding could have been broken.
Criteria for the judgement of ‘high' risk of bias	Any one of the following: no blinding of outcome assessment, and the outcome measurement is likely to be influenced by lack of blinding; blinding of outcome assessment, but likely that the blinding could have been broken, and the outcome measurement is likely to be influenced by lack of blinding.
Criteria for the judgement of ‘unclear' risk of bias	Any one of the following: insufficient information to permit judgement of ‘low' risk or ‘high' risk; the study did not address this outcome.
Incomplete outcome data Attrition bias due to amount, nature or handling of incomplete outcome data
Criteria for a judgement of ‘low' risk of bias	Any one of the following: no missing outcome data; reasons for missing outcome data unlikely to be related to true outcome (for survival data, censoring unlikely to be introducing bias); missing outcome data balanced in numbers across intervention groups, with similar reasons for missing data across groups; for dichotomous outcome data, the proportion of missing outcomes compared with observed event risk not enough to have a clinically relevant impact on the intervention effect estimate; for continuous outcome data, plausible effect size (difference in means or standardised difference in means) among missing outcomes not enough to have a clinically relevant impact on observed effect size; missing data have been imputed using appropriate methods.
Criteria for the judgement of ‘high' risk of bias	Any one of the following: reason for missing outcome data likely to be related to true outcome, with either imbalance in numbers or reasons for missing data across intervention groups; for dichotomous outcome data, the proportion of missing outcomes compared with observed event risk enough to induce clinically relevant bias in intervention effect estimate; for continuous outcome data, plausible effect size (difference in means or standardised difference in means) among missing outcomes enough to induce clinically relevant bias in observed effect size; ‘as‐treated’ analysis done with substantial departure of the intervention received from that assigned at randomisation; potentially inappropriate application of simple imputation.
Criteria for the judgement of ‘unclear' risk of bias	Any one of the following: insufficient reporting of attrition/exclusions to permit judgement of ‘low' risk or ‘high' risk (e.g. number randomised not stated, no reasons for missing data provided); the study did not address this outcome.
Selective reporting Reporting bias due to selective outcome reporting
Criteria for a judgement of ‘low' risk of bias	Any of the following: the study protocol is available and all of the study’s pre‐specified (primary and secondary) outcomes that are of interest in the review have been reported in the pre‐specified way; the study protocol is not available but it is clear that the published reports include all expected outcomes, including those that were pre‐specified (convincing text of this nature may be uncommon).
Criteria for the judgement of ‘high' risk of bias	Any one of the following: not all of the study’s pre‐specified primary outcomes have been reported; one or more primary outcomes was reported using measurements, analysis methods or subsets of the data (e.g. sub scales) that were not pre‐specified; one or more reported primary outcomes were not pre‐specified (unless clear justification for their reporting was provided, such as an unexpected adverse effect); one or more outcomes of interest in the review were reported incompletely so that they could not be entered in a meta‐analysis; the study report fails to include results for a key outcome that would be expected to have been reported for such a study.
Criteria for the judgement of ‘unclear' risk of bias	Insufficient information to permit judgement of ‘low' risk or ‘high' risk. It is likely that the majority of studies will fall into this category.
Other bias Bias due to problems not covered elsewhere in the table
Criteria for a judgement of ‘low' risk of bias	The study appears to be free of other sources of bias.
Criteria for the judgement of ‘high' risk of bias	There is at least one important risk of bias. For example, the study: had a potential source of bias related to the specific study design used; or has been claimed to have been fraudulent; or had some other problem.
Criteria for the judgement of ‘unclear' risk of bias	There may be a risk of bias, but there is either: insufficient information to assess whether an important risk of bias exists; or insufficient rationale or evidence that an identified problem will introduce bias.

Open in a new tab

Data and analyses

Comparison 1. Any intervention to improve obtaining employment compared to another intervention in adults with severe mental illness.

Outcome or subgroup title	No. of studies	No. of participants	Statistical method	Effect size
1 Obtaining competitive employment, short‐term follow‐up (≤ 1 year)	18		Risk Ratio (M‐H, Random, 95% CI)	Subtotals only
1.1 SE vs psych care	3	1087	Risk Ratio (M‐H, Random, 95% CI)	1.79 [0.94, 3.40]
1.2 PVT vs psych care	2	171	Risk Ratio (M‐H, Random, 95% CI)	8.96 [1.77, 45.51]
1.3 TE vs psych care	4	422	Risk Ratio (M‐H, Random, 95% CI)	1.13 [0.88, 1.45]
1.4 SE vs TE	3	231	Risk Ratio (M‐H, Random, 95% CI)	3.49 [1.77, 6.89]
1.5 SE vs PVT	2	148	Risk Ratio (M‐H, Random, 95% CI)	2.52 [1.21, 5.24]
1.6 TE vs PVT	1	89	Risk Ratio (M‐H, Random, 95% CI)	0.78 [0.59, 1.04]
1.7 SE+ vs SE	3	143	Risk Ratio (M‐H, Random, 95% CI)	0.91 [0.37, 2.25]
2 Obtaining competitive employment, long‐term follow‐up (> 1 year)	22		Risk Ratio (M‐H, Random, 95% CI)	Subtotals only
2.1 SE+ vs psych care	1	256	Risk Ratio (M‐H, Random, 95% CI)	4.32 [1.49, 12.48]
2.2 SE vs psych care	1	2238	Risk Ratio (M‐H, Random, 95% CI)	1.51 [1.36, 1.68]
2.3 PVT vs psych care	2	161	Risk Ratio (M‐H, Random, 95% CI)	2.19 [1.07, 4.46]
2.4 SE+ vs TE	2	212	Risk Ratio (M‐H, Random, 95% CI)	2.45 [1.69, 3.55]
2.5 SE vs TE	4	587	Risk Ratio (M‐H, Random, 95% CI)	3.28 [2.13, 5.04]
2.6 SE+ vs PVT	2	193	Risk Ratio (M‐H, Random, 95% CI)	5.42 [1.08, 27.11]
2.7 SE vs PVT	9	1570	Risk Ratio (M‐H, Random, 95% CI)	2.31 [1.85, 2.89]
2.8 SE+ vs SE	3	205	Risk Ratio (M‐H, Random, 95% CI)	1.94 [1.03, 3.65]

Open in a new tab

Comparison 2. Any intervention to improve maintaining employment compared to another intervention in adults with severe mental illness.

Outcome or subgroup title	No. of studies	No. of participants	Statistical method	Effect size
1 Weeks in competitive employment, short‐term follow‐up (≤ 1 year)	8		Mean Difference (IV, Random, 95% CI)	Subtotals only
1.1 SE+ vs SE	1	32	Mean Difference (IV, Random, 95% CI)	‐1.46 [‐3.38, 0.46]
1.2 SE vs TE	2	187	Mean Difference (IV, Random, 95% CI)	4.18 [1.27, 7.09]
1.3 SE vs PVT	1	58	Mean Difference (IV, Random, 95% CI)	6.89 [1.26, 12.52]
1.4 SE vs psych care	2	131	Mean Difference (IV, Random, 95% CI)	4.87 [0.37, 9.37]
1.5 TE vs PVT	1	60	Mean Difference (IV, Random, 95% CI)	6.70 [‐1.76, 15.16]
1.6 TE vs psych care	1	10	Mean Difference (IV, Random, 95% CI)	‐4.39 [‐17.75, 8.97]
2 Weeks in competitive employment, long‐term follow‐up (> 1 year)	11		Mean Difference (IV, Random, 95% CI)	Subtotals only
2.1 SE+ vs SE	3	154	Mean Difference (IV, Random, 95% CI)	10.09 [0.32, 19.85]
2.2 SE+ vs PVT	1	47	Mean Difference (IV, Random, 95% CI)	22.79 [15.96, 29.62]
2.3 SE vs TE	4	587	Mean Difference (IV, Random, 95% CI)	17.36 [11.53, 23.18]
2.4 SE vs PVT	5	390	Mean Difference (IV, Random, 95% CI)	11.56 [5.99, 17.13]

Open in a new tab

Comparison 3. Any intervention to improve obtaining employment compared to another intervention in adults with severe mental illness.

Outcome or subgroup title	No. of studies	No. of participants	Statistical method	Effect size
1 Days to first competitive employment, short‐term follow‐up (≤ 1 year)	3		Mean Difference (IV, Fixed, 95% CI)	Subtotals only
1.1 SE vs TE	1	49	Mean Difference (IV, Fixed, 95% CI)	‐26.60 [‐98.53, 45.33]
1.2 SE vs PVT	1	25	Mean Difference (IV, Fixed, 95% CI)	‐35.94 [‐121.73, 49.85]
1.3 TE vs PVT	1	60	Mean Difference (IV, Fixed, 95% CI)	12.60 [‐23.53, 48.73]
2 Days to first competitive employment, long‐term follow‐up (> 1 year)	6		Mean Difference (IV, Fixed, 95% CI)	Subtotals only
2.1 SE+ vs TE	1	62	Mean Difference (IV, Fixed, 95% CI)	‐142.80 [‐238.70, ‐46.90]
2.2 SE vs TE	3	205	Mean Difference (IV, Fixed, 95% CI)	‐64.86 [‐115.95, ‐13.77]
2.3 SE vs PVT	2	96	Mean Difference (IV, Fixed, 95% CI)	‐35.01 [‐105.21, 35.19]

Open in a new tab

Comparison 4. Any intervention to improve obtaining employment compared to another intervention in adults with severe mental illness.

Outcome or subgroup title	No. of studies	No. of participants	Statistical method	Effect size
1 Obtaining non‐competitive employment, short‐term follow‐up (≤ 1 year)	11		Risk Ratio (M‐H, Random, 95% CI)	Subtotals only
1.1 SE+ vs SE	2	57	Risk Ratio (M‐H, Random, 95% CI)	1.59 [0.15, 17.22]
1.2 SE vs TE	2	187	Risk Ratio (M‐H, Random, 95% CI)	1.10 [0.18, 6.84]
1.3 SE vs PVT	2	148	Risk Ratio (M‐H, Random, 95% CI)	0.89 [0.39, 2.06]
1.4 SE vs psych care	1	900	Risk Ratio (M‐H, Random, 95% CI)	1.10 [0.61, 2.00]
1.5 TE vs PVT	1	89	Risk Ratio (M‐H, Random, 95% CI)	3.82 [2.24, 6.53]
1.6 TE vs psych care	2	78	Risk Ratio (M‐H, Random, 95% CI)	4.27 [0.00, 4883.69]
1.7 PVT vs psych care	1	122	Risk Ratio (M‐H, Random, 95% CI)	3.0 [0.12, 72.23]
2 Obtaining non‐competitive employment, long‐term follow‐up (> 1 year)	12		Risk Ratio (M‐H, Random, 95% CI)	Subtotals only
2.1 SE+ vs TE	2	212	Risk Ratio (M‐H, Random, 95% CI)	0.44 [0.12, 1.66]
2.2 SE+ vs psych care	1	256	Risk Ratio (M‐H, Random, 95% CI)	44.69 [6.25, 319.49]
2.3 SE vs TE	4	587	Risk Ratio (M‐H, Random, 95% CI)	0.22 [0.08, 0.63]
2.4 SE vs PVT	4	582	Risk Ratio (M‐H, Random, 95% CI)	1.29 [0.47, 3.53]
2.5 SE vs psych care	1	2238	Risk Ratio (M‐H, Random, 95% CI)	1.04 [0.76, 1.40]

Open in a new tab

Comparison 5. Any intervention to improve obtaining or maintaining employment compared to another intervention in adults with severe mental illness.

Outcome or subgroup title	No. of studies	No. of participants	Statistical method	Effect size
1 Quality of life, long‐term follow up (> 1 year)	9		Std. Mean Difference (IV, Fixed, 95% CI)	Subtotals only
1.1 SE+ vs psych care (QOLI)	1	256	Std. Mean Difference (IV, Fixed, 95% CI)	‐0.79 [‐1.05, ‐0.54]
1.2 SE vs psych care (QOLI)	1	2238	Std. Mean Difference (IV, Fixed, 95% CI)	0.14 [0.06, 0.23]
1.3 SE vs TE (QOLI)	2	352	Std. Mean Difference (IV, Fixed, 95% CI)	0.05 [‐0.16, 0.26]
1.4 SE+ vs SE (PWI)	1	114	Std. Mean Difference (IV, Fixed, 95% CI)	0.33 [‐0.04, 0.70]
1.5 SE vs TE (W‐QLI objective)	1	100	Std. Mean Difference (IV, Fixed, 95% CI)	‐0.14 [‐0.53, 0.26]
1.6 SE vs TE (W‐QLI subjective)	1	100	Std. Mean Difference (IV, Fixed, 95% CI)	‐0.10 [‐0.50, 0.29]
1.7 SE+ vs PVT (PWI)	1	124	Std. Mean Difference (IV, Fixed, 95% CI)	0.41 [0.06, 0.77]
1.8 SE vs PVT (PWI)	1	131	Std. Mean Difference (IV, Fixed, 95% CI)	0.14 [‐0.20, 0.48]
1.9 SE vs PVT (MANSA)	2	369	Std. Mean Difference (IV, Fixed, 95% CI)	0.14 [‐0.06, 0.35]
1.10 SE vs PVT (QOLP)	1	312	Std. Mean Difference (IV, Fixed, 95% CI)	0.0 [‐0.22, 0.22]

Open in a new tab

Comparison 6. Any intervention to improve obtaining or maintaining employment compared to another intervention in adults with severe mental illness.

Outcome or subgroup title	No. of studies	No. of participants	Statistical method	Effect size
1 Mental health long‐term follow‐up (> 1 year)	7		Mean Difference (IV, Random, 95% CI)	Subtotals only
1.1 SE vs psych care (SFHS)	1	2238	Mean Difference (IV, Random, 95% CI)	2.88 [1.78, 3.98]
1.2 PVT vs psych care (PANSS positive symptoms)	1	103	Mean Difference (IV, Random, 95% CI)	‐2.48 [‐3.95, ‐1.01]
1.3 PVT vs psych care (PANSS negative symptoms)	1	103	Mean Difference (IV, Random, 95% CI)	‐1.61 [‐2.99, ‐0.23]
1.4 PVT vs psych care (PANSS general symptoms)	1	103	Mean Difference (IV, Random, 95% CI)	‐1.86 [‐3.09, ‐0.63]
1.5 PVT vs psych care (composite index, multiple scales)	1	58	Mean Difference (IV, Random, 95% CI)	‐7.09 [‐326.22, 312.04]
1.6 SE vs TE (PANSS positive symptoms)	1	200	Mean Difference (IV, Random, 95% CI)	‐0.02 [‐1.71, 1.67]
1.7 SE vs TE (PANSS negative symptoms)	1	200	Mean Difference (IV, Random, 95% CI)	‐0.80 [‐2.79, 1.19]
1.8 SE vs TE (PANSS general psychopathology)	1	200	Mean Difference (IV, Random, 95% CI)	‐2.69 [‐7.58, 2.20]
1.9 SE vs TE (BPRS)	1	152	Mean Difference (IV, Random, 95% CI)	‐1.90 [‐5.69, 1.89]
1.10 SE vs PVT (HADS anxiety)	1	312	Mean Difference (IV, Random, 95% CI)	0.10 [‐0.88, 1.08]
1.11 SE vs PVT (HADS depression)	1	312	Mean Difference (IV, Random, 95% CI)	‐0.10 [‐1.08, 0.88]
1.12 SE vs PVT (MHI)	1	150	Mean Difference (IV, Random, 95% CI)	‐1.00 [‐8.63, 2.63]
1.13 SE vs PVT (PANSS positive symptoms)	1	312	Mean Difference (IV, Random, 95% CI)	0.10 [‐0.93, 1.13]
1.14 SE vs PVT (PANSS negative symptoms)	1	312	Mean Difference (IV, Random, 95% CI)	‐0.20 [‐1.38, 0.98]
1.15 SE vs PVT (PANSS general psychopathology)	1	312	Mean Difference (IV, Random, 95% CI)	0.40 [‐1.34, 2.14]

Open in a new tab

Comparison 7. Any intervention to improve obtaining or maintaining employment compared to another intervention in adults with severe mental illness.

Outcome or subgroup title	No. of studies	No. of participants	Statistical method	Effect size
1 Dropouts, short‐term follow‐up (≤ 1 year)	13		Risk Ratio (M‐H, Fixed, 95% CI)	Subtotals only
1.1 SE vs psych care	3	1087	Risk Ratio (M‐H, Fixed, 95% CI)	0.67 [0.41, 1.10]
1.2 PVT vs psych care	1	122	Risk Ratio (M‐H, Fixed, 95% CI)	5.0 [0.25, 102.04]
1.3 TE vs psych care	2	182	Risk Ratio (M‐H, Fixed, 95% CI)	0.67 [0.43, 1.06]
1.4 SE vs TE	2	187	Risk Ratio (M‐H, Fixed, 95% CI)	1.78 [0.84, 3.77]
1.5 SE vs PVT	2	148	Risk Ratio (M‐H, Fixed, 95% CI)	1.36 [0.56, 3.30]
1.6 TE vs PVT	1	89	Risk Ratio (M‐H, Fixed, 95% CI)	3.92 [0.19, 79.40]
1.7 SE+ vs SE	2	119	Risk Ratio (M‐H, Fixed, 95% CI)	0.85 [0.43, 1.67]
2 Dropouts, long‐term follow‐up (> 1 year)	19		Risk Ratio (M‐H, Fixed, 95% CI)	Subtotals only
2.1 SE+ vs psych care	1	256	Risk Ratio (M‐H, Fixed, 95% CI)	0.74 [0.50, 1.08]
2.2 SE vs psych care	1	2238	Risk Ratio (M‐H, Fixed, 95% CI)	1.09 [0.78, 1.52]
2.3 PVT vs psych care	2	161	Risk Ratio (M‐H, Fixed, 95% CI)	0.99 [0.45, 2.19]
2.4 SE+ vs SE	1	123	Risk Ratio (M‐H, Fixed, 95% CI)	0.65 [0.37, 1.14]
2.5 SE+ vs TE	1	143	Risk Ratio (M‐H, Fixed, 95% CI)	0.61 [0.33, 1.13]
2.6 SE vs TE	4	587	Risk Ratio (M‐H, Fixed, 95% CI)	1.92 [0.89, 4.15]
2.7 SE+ vs PVT	2	193	Risk Ratio (M‐H, Fixed, 95% CI)	0.99 [0.59, 1.64]
2.8 SE vs PVT	9	1569	Risk Ratio (M‐H, Fixed, 95% CI)	0.95 [0.80, 1.13]

Open in a new tab

Comparison 8. Any intervention to improve obtaining or maintaining employment compared to another intervention in adults with severe mental illness.

Outcome or subgroup title	No. of studies	No. of participants	Statistical method	Effect size
1 Hospital admissions, short‐term follow‐up (≤ 1 year)	6		Risk Ratio (M‐H, Random, 95% CI)	Subtotals only
1.1 SE vs psych care	1	900	Risk Ratio (M‐H, Random, 95% CI)	1.13 [0.74, 1.73]
1.2 TE vs psych care	4	422	Risk Ratio (M‐H, Random, 95% CI)	0.59 [0.30, 1.15]
1.3 SE vs PVT	1	90	Risk Ratio (M‐H, Random, 95% CI)	1.24 [0.76, 2.01]
2 Hospital admissions, long‐term follow‐up (> 1 year)	6		Risk Ratio (M‐H, Random, 95% CI)	Subtotals only
2.1 SE+ vs psych care	1	256	Risk Ratio (M‐H, Random, 95% CI)	1.11 [0.73, 1.70]
2.2 PVT vs psych care	1	103	Risk Ratio (M‐H, Random, 95% CI)	0.26 [0.11, 0.65]
2.3 SE+ vs TE	1	143	Risk Ratio (M‐H, Random, 95% CI)	0.94 [0.55, 1.63]
2.4 SE vs PVT	3	681	Risk Ratio (M‐H, Random, 95% CI)	0.93 [0.60, 1.45]

Open in a new tab

Characteristics of studies

Characteristics of included studies [ordered by study ID]

Au 2015.

Methods	Allocation: randomised Design: multi centre Duration: 11 months Country: Hong Kong, China
Participants	N = 90 Diagnosis: schizophrenia or schizoaffective disorder as confirmed by the Chinese Version Structural Clinical Interview for the DSM IV. Included: 58% schizophrenia, 42% schizoaffective disorder Setting: recruitment of patients from two local psychiatric outpatient clinics or day hospitals Age: ≥ 18 years, mean 36.1 years Gender: 63% male Ethnicity: "Chinese people" Substance abuse: ‐ Living situation: ‐ Marital status: 88% single Employment status: unemployed Working history: 97% employment history Motivation: competitive employment as their current vocational goal Education: mean 15 years Disability benefit: ‐ Excluded: moderate or greater cognitive impairment. Excluded participants that had a score of > 18 on the 30‐item Mini‐Mental State Examination and/or were not mentally capable of giving informed consent
Interventions	Integrated supported employment (N = 45) 6/7 core features of the IPS were incorporated with the exception of the rapid job search. Instead, ten WSST sessions (1.5–2 h/week) were conducted in group format prior to job search. Individualised ongoing support was given on an unlimited time basis within the study period after participants obtained employment. Integrated supported employment + cognitive remediation training (N = 45) Participants in the ISE + CRT programme received 6 h/week of individualised, visual‐based computer‐assisted cognitive exercises by 2 cognitive remediation software systems (Strong arm system and Captain's Log). A TV‐watching session was added on top of the ISE group as a control to neutralise the effect of additional time and therapist contact due to CRT in the ISE + CRT group.
Outcomes	Percentage of participants who obtained competitive employment Weeks in competitive employment Mental health (BPRS) Quality of life (PWI) Drop outs
Notes	Success in competitive employment was defined as having continuously worked in a job for ≥ 2 months for at least 20h/week No IPS/SE fidelity measurements reported Not included in the network meta‐analysis and direct comparison meta‐analysis because this is the only study about this intervention (comparison)
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	"Randomly assigned by a blinded research assistant based on random assignment generated by SPSS"
Allocation concealment (selection bias)	Low risk	Assignment by a blinded research assistant
Blinding of participants and personnel (performance bias)   All outcomes	High risk	Participants and personnel could identify given intervention by contents of programme
Blinding of outcome assessment (detection bias)   All outcomes	Low risk	"Blinded assessments on outcomes were conducted by independent assessors"
Incomplete outcome data (attrition bias)   All outcomes	Low risk	The programme attrition rate was 11%, with 9% for the intervention group and 13% for the control group. All participants were included in analyses following the ‘Intent‐to‐treat’ principle, with the last observation carried forward to replace any missing data.
Selective reporting (reporting bias)	Low risk	All listed outcomes reported
Other bias	Low risk	This study was funded by a grant from the Health and Medical Research Fund Committee (formally Health and Health Services Research Fund; HHSRF Project No.: 08091201). The content is solely the responsibility of the study authors and does not necessarily represent the official views of the funding agencies

Open in a new tab

Beard 1963.

Methods	Allocation: randomised Design: single centre Duration: 2 years (preliminary data for participants who had completed at least 1 year of study) Country: New York City, USA
Participants	N = 212 (total sample N = 352) Diagnosis: in the community for < four months and previous hospitalisation of ≥ 2 months. Included: 75% schizophrenia, 7% other psychotic disorders, 11% psychoneurosis or depressive reaction Setting: Fountain House Foundation, a psychiatric rehabilitation centre Age: 68% under 35 years Gender: 60% male Ethnicity: 88% white Substance abuse: ‐ Living situation: ‐ Marital status: 70% never married Employment status: 92% unemployed Working history: ‐ Motivation: ‐ Education: 14% graduated from college, 30% attended college and 60% high school graduates Disability benefit: ‐ Excluded: out of hospital and in the community for ≥ 2 years, hospitalised primarily for drug addiction, alcoholism, overt homosexuality, uncontrolled epilepsy and criminal behaviour
Interventions	Fountain House (N = 163) A programme of social and recreational activities to rebuild confidence, self‐esteem and social skills. In addition, there were day activities focused on work‐ordered activities in work crews in and around the Fountain House. Also, participants took part in supported employment for 4 months, after completing the programme of social and recreational activities and participating in work crews. After having completed those 4 months successfully participants would go on to obtain regular jobs. Control (N = 49) The control group represented those individuals who would not otherwise have received services, due to lack of facilities and personnel. They continued to receive community care from other services.
Outcomes	Percentage of participants in competitive employment Hospital admissions
Notes	A person was considered "gainfully employed", regardless of the number of h/week he worked Employment outcomes after 12 months (4 quarters)
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	High risk	To achieve random assignment of research participants to the control and experimental groups, a method of "rotation at application," based on the time and date of the participant's application, was used. For every participant assigned to the control group, the next 3 were consecutively assigned to the experimental condition
Allocation concealment (selection bias)	High risk	Allocation was not concealed due to method of rotation at application
Blinding of participants and personnel (performance bias)   All outcomes	High risk	Participants and personnel could identify the given intervention by contents of the programme
Blinding of outcome assessment (detection bias)   All outcomes	High risk	No details
Incomplete outcome data (attrition bias)   All outcomes	High risk	Follow‐up information on participants was secured on approximately 80% of control participants and 88% of experimental participants. Not all participants were followed up for a full 2 years. Participants continued to enter the study until the last 3 months. Thus numbers followed up are different at different time points.
Selective reporting (reporting bias)	High risk	Employment outcomes are only available in preliminary report
Other bias	Low risk	National Institute of Mental Health Project Grant OM‐29 1(RI)(C1)

Open in a new tab

Becker 1967.

Methods	Allocation: randomised Design: single centre Duration: 8 months Country: Fort Worth, Texas, USA
Participants	N = 50 Diagnosis: chronically hospitalised psychiatric patients. Included: 78% schizophrenia, 14% chronic brain syndrome, 8% severe neurosis or character disorder Setting: Public Health Service Hospital, a federal hospital specialising in the treatment of narcotic addicts. The hospital also treats general psychiatric patients on a separate 500 bed service. Age: mean 46 years Gender: ‐ Ethnicity: ‐ Substance abuse: ‐ Living situation: hospitalised Marital status: ‐ Employment status: ‐ Working history: the majority of participants had been employed before hospitalisation as unskilled labourers Motivation: ‐ Education: very few had a high school diploma, 1/3 less than 7th grade Disability benefit: ‐ Excluded: > 62 years, hospitalised < 2 years in last 4 years, physically disabled‐bedridden, discharge plans completed, unpredictable physical violence, disabling organicity, hospital general psychiatric census
Interventions	Experimental rehabilitation (N = 25) This was a specialised rehabilitation ward, where intensive multi‐disciplinary input, social skills groups, and group and individual vocational assignments were given. In addition, tours of local industrial facilities, sheltered workshop, and transitional work experience in local community enterprises were arranged. The most important aspect of this service was organised interagency co‐operative management of participants in community sheltered employment. Traditional continued treatment programmes (N= 25) Continuation of inpatient treatment on rehabilitation wards, option of referral to external VR services
Outcomes	Percentage of participants who obtained competitive employment Percentage of participants who obtained non‐competitive employment Hospital admissions Dropouts
Notes	We will only use data from phase I: after phase I all participants became a new intervention group in phase II, in phase III all participants were randomised again Competitive employment was not specified
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	Two groups were selected by lot. No further details
Allocation concealment (selection bias)	Unclear risk	No details
Blinding of participants and personnel (performance bias)   All outcomes	High risk	Participants and personnel could identify given intervention by content of program
Blinding of outcome assessment (detection bias)   All outcomes	High risk	No details
Incomplete outcome data (attrition bias)   All outcomes	Low risk	1 of 50 participants lost to follow‐up (suicide)
Selective reporting (reporting bias)	Low risk	All listed outcomes reported
Other bias	Unclear risk	No details about funding source

Open in a new tab

Bejerholm 2015.

Methods	Allocation: randomised Design: multicentre Duration: 18 months Country: Malmo, Sweden
Participants	N = 120 Diagnosis: SMI, which refers to having a psychosis diagnosis or a psychiatric diagnosis where the psychiatric disabilities significantly impact on everyday life functioning on a long‐term basis (2 years) Included: 64% psychotic disorder, 8% bipolar, 28% other Setting: participants were recruited from all 6 mental health teams in a southern Swedish city. Age: 18‐63, mean 38 years Gender: 56% male Ethnicity: 64% native, 36% immigrant Substance abuse:‐ Living situation:‐ Marital status: 82% single Employment status: had not worked in the preceding year Working history: 56% worked in the last 5 years Motivation: desire to work in the near future Education: ‐ Disability benefit: figures from a region close to where the present RCT took place showed that 9/10 participants were relying on sick leave benefits for their income Excluded: a somatic comorbidity causing reduced work ability
Interventions	IPS (N = 60) The 8 principles of IPS were administrated by the employment specialist, and were adhered to 3 employment specialists were recruited. Their caseload for working full time was 20 participants. The IPS service was integrated with the mental healthcare service sharing the same facilities as the teams. Continuous information and discussion meetings were held 8 months before the start and throughout the study together with 6 mental healthcare teams, both national and private, the Social Insurance Agency (SIA), the Public Employment Service (PES), and FINSAM, a state‐funded organisation to facilitate co‐ordination across the healthcare system, municipality, SIA and PES. Furthermore, workshops were arranged in relation to the IPS fidelity evaluations. The fidelity score at 6 months was 110 (good fidelity), at 12 months 115 (excellent fidelity) and at 18 months 117 points (excellent fidelity) TVR (N = 60) 'Train‐place' vocational services located in the four welfare organisations, the healthcare, municipality, SIA and the PES. Typically, these nationally‐run services provide PVT in sheltered settings in a stepwise manner. The allocation of participants was dependent on the individuals’ care needs and symptom severity, as estimated by professionals in the mental healthcare team. The services ranged from individual rehabilitation support from a team member in the mental healthcare service, most often occupational therapists (50% of the participants), municipality‐run sheltered or day centre activities and PVT, joint co‐operation of vocational service in the SIA/PES, and support from either the PES or the SIA. Some participants also enrolled themselves in Fountain House (clubhouse) activities.
Outcomes	Percentage of participants who obtained competitive employment Percentage of participants who obtained non‐competitive employment Weeks in competitive employment Days to first competitive employment Dropouts Quality of life (MANSA)
Notes	All competitively employed worked for at least 1 week in employment that paid at least minimum wage, available to any citizen and located in mainstream settings
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	"The randomisation was done centrally at the Swedish Institute of Health Sciences. The software programme in use produced a randomisation plan covering a block size of 8 random group allocation numbers at a time"
Allocation concealment (selection bias)	Low risk	Central allocation
Blinding of participants and personnel (performance bias)   All outcomes	High risk	"It was not possible to mask the participants’ allocation status for the study participants and the professionals involved after randomisation"
Blinding of outcome assessment (detection bias)   All outcomes	Low risk	"The researchers had no previous knowledge of the identity of any participant and coded data." "The allocation status was assessor‐blinded"
Incomplete outcome data (attrition bias)   All outcomes	High risk	73% follow up. They performed a power analysis prior to the study and an ITT data analysis. ITT data are presented for best case or worst case of the primary outcome (obtaining competitive employment) scenario with imputation. Reasons for dropouts are reported
Selective reporting (reporting bias)	Low risk	All listed outcomes are reported
Other bias	Low risk	Financial support for this study was provided by a grant from the Swedish Research Council for Health, Working life and Welfare, and FINSAM. The Medical Faculty of Lund University and Vårdal Institutet contributed with the funding of researchers’ and research assistants’ wages

Open in a new tab

Blankertz 1996.

Methods	Allocation: randomised Design: single centre Duration: 9 months Country: Philadelphia, Pennsylvenia, USA
Participants	N = 122 Diagnosis: severe mental illness. Included: 72% schizophrenia, 25% major affective disorder Setting: community mental health centre operating partial hospital programmes and outpatient programmes Age: mean 36 years Gender: 64% male Ethnicity: 80% white Substance abuse: 60% reported use of alcohol or street drugs Living situation: ‐ Marital status: 84% never married Employment status: unemployed, mean duration 9 years Working history: 82%, the positions included dishwasher, labourer, janitor, retail salesperson, mean length of employment 1 year Motivation: ‐ Education: mean years 12, 59% high school diploma, 17% some type of college degree Disability benefit:‐ Excluded: ‐
Interventions	Work focused programme (N = 61) A variety of rehabilitation intervention techniques, including one‐on‐one meetings, group sessions, individual advocacy, and long‐term supports, were used. While participating in the work‐focused programme, regular services from the CMH centre were provided, including partial hospital or outpatient services, case management, therapy, and medication monitoring. Programme interventions were based on techniques compatible with social learning theory, such as helping the client set attainable subgoals based on skill attainment and providing positive reinforcement for reaching these goals, and expectancy theory, in which motivation to work is seen as a function of positive valuation of work, the possession of necessary skills, and self‐efficacy. Control (N =61) Standard services offered, including partial hospitalisation, outpatient services, case management if needed, individual therapy, without specific vocational focus. Therapist typically provided little support to clients who were applying to participate in the system
Outcomes	Percentage of participants who obtained competitive employment Percentage of participants who obtained non‐competitive employment Drop outs
Notes	Competitive employment was not specified
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	Participants were randomly assigned
Allocation concealment (selection bias)	Unclear risk	No details
Blinding of participants and personnel (performance bias)   All outcomes	High risk	Participants and personnel could identify the given intervention by contents of the programme
Blinding of outcome assessment (detection bias)   All outcomes	High risk	No details
Incomplete outcome data (attrition bias)   All outcomes	Unclear risk	The authors suggested that outpatient clients assigned to the experimental group tended to drop out in the first week after randomisation. However, they did not report the exact numbers. They only mentioned that 2 clients had left the experimental group after the first week and none in the control group. It is possible that this is not an ITT analysis.
Selective reporting (reporting bias)	High risk	Preliminary data, we did not find other reports
Other bias	Unclear risk	It was funded as a research and demonstration project by the National Institute on Disability and Rehabilitation Research. Those in the experimental group were significantly more likely to attend the partial hospital programme than those in the control group. Outpatient clients did not want to participate in the experimental group because they felt they were at a different functioning level, that is they felt that clients in the partial programme needed more structured programming and were less likely to have the skills needed for independent living. The group also differed in employment history.

Open in a new tab

Bond 1986.

Methods	Allocation: randomised Design: 2 x 2 factorial design Duration: 15 months Country: Chicago, Illinois, USA
Participants	N = 131 Diagnosis: 55% schizophrenia, 19% affective disorder and 26% personality disorder Setting: a private psychosocial rehabilitation agency in the inner‐city area of a large Midwestern city Age: ≥ 18 years, mean 24.5 years Gender: 69% male Ethnicity: 75% white Substance abuse: ‐ Living situation: 25% living in hospital at the time of admission, 47% lived with parents or relatives, 14% lived in halfway houses, 2% lived sheltered, 14% lived alone Marital status: ‐ Employment status: unemployed Working history: 54% < 1 year work experience, 46% ≥ 1 year, 18% never worked for a period of > 3 months Motivation: stated goal of future employment Education: 80% high school graduates Disability benefit: 61% used government assistance Excluded: primary diagnosis of substance abuse or developmental disability, no prior participation in the programme of more than 30 days
Interventions	Accelerated programme (N = 64) Participants in this programme immediately began a paid group placement for a minimum of 2 d/week. Thereafter, they were not to be returned to their prevocational crew, only if a strong justification was given by their caseworkers. Gradual programme ( N = 67) A series of graded work experiences intended to prepare members for competitive employment. Members began in prevocational crew, moved on to the group placement after 3‐6 months, them moved on to individual placement and finally moved to their own jobs. Participants in the gradual condition remained in the work crews for a minimum of 4 months. They were also discouraged from seeking community employment.
Outcomes	Percentage of participants who obtained competitive employment Percentage of participants who obtained non‐competitive employment Hospital admissions Dropouts
Notes	Definition of competitive employment: employed at end of interval minimal 6 h/week Not included in the network meta‐analyses and direct comparison meta‐analyses, because we could not classify these interventions in different groups
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	This study employed a 2 x 2 factorial design. Members were classified as work‐experienced or work inexperienced. They were then randomly assigned to 1 of 2 vocational conditions
Allocation concealment (selection bias)	Low risk	"After randomisation the research assistant contacted the caseworker who in turn met with the client to explain the assignment"
Blinding of participants and personnel (performance bias)   All outcomes	High risk	Participants and personnel could identify given intervention by contents of programme
Blinding of outcome assessment (detection bias)   All outcomes	High risk	No details about blinding. Participants were interviewed, computerised hospital records based on information provided by caseworkers were used
Incomplete outcome data (attrition bias)   All outcomes	High risk	Data were collected on 82% at 15‐month follow‐up (N = 57 vs N = 50). N = 22 in the intervention group and N = 13 in the control group failed on placements or dropped out. 61% of the intervention group and 52% of the control group terminated from the agency during 15 months' follow‐up
Selective reporting (reporting bias)	Low risk	All listed outcomes were reported
Other bias	Unclear risk	This research was supported by a grant from the Field Foundation and by Grant No. 8103 from the Illinois Department of Mental Health and Developmental Disabilities. Portion of this research was completed when Gary Bond was Director of Research at Thresholds

Open in a new tab

Bond 1995.

Methods	Allocation: randomised Design: multicentre Duration: 4 years Country: Indianapolis, Indiana, USA
Participants	N = 86 Diagnosis: serious mental illness according to Indiana Department of Mental Health criteria (major mental disorder and demonstrated disability and duration) Included: 66% schizophrenia or schizoaffective disorder, 14% personality disorder, 11% affective disorder, 17% secondary diagnosis of developmental disability Setting: 2 rehabilitation agencies of which one was a CMHC distributed over 4 centres located in an urban‐rural distribution and the other was a private not‐for‐profit agency located in Indianapolis. Age: 18 ‐60 years, mean 53.1 years Gender: 51% male Ethnicity: 80% white Substance abuse: 22% substance abuse problems Living situation: ‐ Marital status: 84% never married Employment status: unemployed for past 3 months, mean time since last job 38 months Working history: 70% had been employed continuously for a year or longer in a competitive job Motivation: expressing a desire to obtain competitive employment Education: 59% high school degree Disability benefit: being a recipient of or judged eligible for SSDI or SSI: 57% SSDI or combination, 28% SSI only Excluded: no formal vocational training in past 6 months
Interventions	Accelerated entry into supported employment (N = 43) An immediate start in the SE programme after study admission, consisting of focusing on immediate competitive employment, without PVT. No screening of participants who were assumed suitable for employment. The clients’ strengths and preferences were evaluated to find jobs matching the client. In addition, the programme helped locating jobs through systematic contact with employers and occasionally negotiated with the employer to make reasonable accommodation. In addition, clients were given extensive job coaching after placement and follow‐along support was continued indefinitely. Non‐vocational aspects of their rehabilitation and treatment were also available. Gradual entry intro SE (N = 43) A minimum of 4 months’ preparation in prevocational work readiness training, before being eligible for the SE programme, consisting of vocational readiness classes, which taught skills such as resume writing, job interviewing and job keeping. After this the SE programme was started.
Outcomes	Percentage of participants who obtained competitive employment Weeks in competitive employment Days to first competitive employment Dropouts
Notes	Definition competitive employment: whether or not a client was competitively employed during follow‐up We used data after 1 year for this review, because employment outcomes after 2 and 4 years were only those who were currently employed
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	Clients were randomly assigned
Allocation concealment (selection bias)	Unclear risk	No details
Blinding of participants and personnel (performance bias)   All outcomes	High risk	Participants and personnel could identify given intervention by contents of the programme
Blinding of outcome assessment (detection bias)   All outcomes	High risk	"The employment specialists were responsible for providing all research data to the research team"
Incomplete outcome data (attrition bias)   All outcomes	High risk	Outcome data at 1 year follow‐up available for 86% of the participants (N = 39 and N = 35). Reasons for missing data reported. Final sample excluding 4‐month dropouts N = 65 (N = 34 and N = 31). No further details. They did not perform an ITT analysis
Selective reporting (reporting bias)	Low risk	All listed outcomes were reported
Other bias	Low risk	Support for this study was provided by Social Security Administration Grant 12‐D‐70299‐5‐01. The first author was supported by the Research Scientist Development Award K02 MH 00842 from the National Institute of Mental Health

Open in a new tab

Bond 2007.

Methods	Allocation: randomised Design: multisite Duration: 24 months Country: Chicago, Illinois, USA
Participants	N = 200 Diagnosis: severe mental illness according to State of Illinois criteria, i.e. DSM IV criteria of schizophrenia spectrum disorder, bipolar disorder, obsessive‐compulsive disorder or other psychotic disorder and either significant treatment history or significant functional impairments (39% schizophrenia, 17% schizoaffective disorder, 24% bipolar disorder, 17% depression) Setting: Thresholds, psychiatric rehabilitation service  Age: ≥ 18 years, mean 38.8 years  Gender: 64% male  Ethnicity: 51% African American, 38% white, 8% Hispanic Substance use: 32% used alcohol and 21% used drugs during study Living situation: 35% independent, 28% with family, 13% semi‐independent, 21% institutional, 3% homeless Marital status: 74% single, 21% divorced, 4% married Employment status: unemployed, no competitive employment in past 30 days Motivation: expressed goal of paid employment Work history: 40% ≥ 1 year work experience Education: 45% some college or associate's, 26% high school graduate or GED, 18% did not graduate high school Disability benefit: 76%  Excluded: physical illness that would likely prevent participation throughout course of full 2 years of the study
Interventions	IPS (N = 100) The IPS programme was newly implemented. The programme model followed closely that described in the IPS manual. The implementation went poorly during the first 6 months. Programme fidelity (IPS fidelity scale) scores were low in the first 6 months but after personnel changes, the fidelity equalled or exceeded 70 during the rest of the study. All ratings were made by a single assessor (one of the authors) DPA (N = 100) DPA was a Thresholds vocational programme, that adhered to Clubhouse values concerning client empowerment but departed from the Cubhouse vocational programme standards. It is a stepwise approach to competitive employment in which clients are assessed on work readiness during PVT, then typically are initially placed in protected jobs for and indefinite period of time. DPA offers a range of job options from an existing pool of placements available through agency‐run businesses and standing relationships with employers. DPA emphasises peer support by maximising the clubhouse environment, group placements and employment groups.
Outcomes	Percentage of participants who obtained competitive employment Number of weeks in competitive employment Number of days to first competitive employment Percentage of participants who obtained non‐competitive employment Mental health (PANNS) Quality of life (QOLI) Dropouts
Notes	Competitive employment was defined as a job with a community employer in an integrated community setting, paying at least minimum wage.
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	"Randomisation was achieved by means of a computerised randomisation list in lots of 20 at each site"
Allocation concealment (selection bias)	Low risk	"After the baseline interview was completed, the interviewer called the offsite project director to report the identification number, programme location, and work history (experienced/inexperienced). The project director responded with study condition as determined by an a priori computerised randomisation list prepared for each work history level within site"
Blinding of participants and personnel (performance bias)   All outcomes	High risk	Participants and personnel could identify the given intervention by contents of the programme
Blinding of outcome assessment (detection bias)   All outcomes	High risk	"Interviewers were not blind to vocational programme assignment"
Incomplete outcome data (attrition bias)   All outcomes	High risk	Number of lost to follow‐up reported, but no reasons for missing data provided. Of the 200 participants 171 (85.5%) completed the 24‐month interview. From 16 other participants employment data were obtained, yielding 187 participants (93.5%) in the final sample (IPS N = 92, DPA N = 95). Treatment exposure sample IPS n = 78 and DPA n = 64
Selective reporting (reporting bias)	Low risk	All listed outcome of interest were reported
Other bias	Low risk	Funding: supported by Grant R01MH59987 from the National Institute of Mental Health. No details. No evidence of other bias.

Open in a new tab

Bond 2015b.

Methods	Allocation: randomised Design: multi‐site Duration: 12 months Country: Chicago, Illinois, USA
Participants	N = 90 Diagnosis: severe mental illness according to state criteria that is, diagnosis of schizophrenia spectrum disorder, bipolar disorder, or other psychotic disorder and either significant treatment history or significant functional impairments (53% schizophrenia, 18% depressive disorder, 25% bipolar disorder) Setting: Thresholds, a large psychiatric rehabilitation agency Age: ≥ 18 years, mean 43.8 years Gender: 79% male Ethnicity: 59% African American, 30% white Substance use: 22% alcohol dependence or abuse, 30% drug dependence or abuse. Living situation: 57% own apartment, 22% group home Marital status: 69% never married, 18% divorced Employment status: no competitive employment in past 3 months Work history: 49% worked in past 5 years, 95% held a competitive job in the past Motivation: expressed interest in a competitive job Education: 40% less than high school, 24% high school graduate, 32% some college, 3% college graduate. Criminal justice involvement: 98% arrested in the past, 76% incarcerated, 50% drug offence 44% theft, 37% violence, 62% felony, 67% misdemeanour, 20% arrested during study Disablity benefit: 89% Excluded: no legal, physical or other restriction that would prevent participating over the 12 months' follow‐up period, including pending criminal charges
Interventions	IPS (N = 45) The IPS condition followed the principles of IPS SE, enhanced with a day‐long training for IPS employment specialists on criminal justice issues. The fidelity was assessed with the Revised Individual Placement and Support Fidelity Scale (IPS‐25). All fidelity reviews met the criteria for good fidelity. Work Choice (N = 45) Work Choice was based on the empirically validated job club model, tailored for people with psychiatric disabilities. It facilitated a self‐directed job search, helping clients with resume preparation, interview skills, and job leads. Classes were scheduled weekly at two conveniently located sites. The curriculum included training in application procedures, job search strategies, and linkage services. The classes were held in a room with computer workstations for applying online for jobs. A 14‐item Work Choice fidelity scale was developed for the study. The total score was 4.6, indicating adequate fidelity.
Outcomes	Percentage participants who obtained competitive employment Percentage of participants in non‐competitive employment Hospital admissions Dropouts
Notes	Competitive employment defined as jobs in integrated work settings in the competitive job market at prevailing wages, with supervision provided by personnel employed by the business
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	"A biostatistician prepared a randomised list based on an urn randomisation technique with block size equal to four"
Allocation concealment (selection bias)	Low risk	"Each participant opened the next consecutively numbered, sealed envelope, which revealed the assigned study condition"
Blinding of participants and personnel (performance bias)   All outcomes	High risk	Participants and personnel could identify the given intervention by contents of the programme
Blinding of outcome assessment (detection bias)   All outcomes	High risk	Participants were interviewed and employment outcomes were corroborated through the agency's management information system and employment specialist logs. Blinding is not described, but it is unlikely.
Incomplete outcome data (attrition bias)   All outcomes	Low risk	Main outcome data were available for 85 participants (94%)
Selective reporting (reporting bias)	Low risk	All listed outcomes were reported
Other bias	Low risk	Baseline difference has been taken into account This study was supported by grant H133G100110 from the National Institute of Disability and Rehabilitation Research.

Open in a new tab

Burns 2007.

Methods	Allocation: randomised  Design: multicentre  Duration: 18 months  Country: several European countries: London, UK; ULM‐Guenzburg, Germany; Rimini, Italy; Zurich, Switzerland; Groningen, Netherlands; Sofia, Bulgaria
Participants	N = 312 Diagnosis: severe mental illness (80% schizophrenia or schizoaffective disorder, 17% bipolar disorder), had been ill and had major role dysfunction for at least 2 years Setting: 6 European mental health centres Age: between 18 years and local retirement age, mean 37.8 years Gender: 60% male Ethnicity: 90% born in country of residence Substance abuse: ‐ Living situation: 52% with friends/relatives, 34% alone, 14% sheltered Marital status: not reported Employment status: had not been in competitive employment in the preceding year Work history: 56% worked more than 1 month in past 5 years Motivation: wished to enter competitive employment Education: mean number of years 11.9 Disability benefit: ‐ Excluded: ‐
Interventions	IPS (N = 156) The IPS model consists of identification of patients who want to work in the competitive labour market, and helps them develop realistic goals and seek appropriate employment directly; there is no training phase. The IPS worker builds up a network of employers willing to accept patients, with whom the IPS worker continues contact, supporting both patient and employer. This support is open ended (in our study until the end of the 18‐month follow‐up), and the IPS worker had a maximum caseload of 25 patients. When the local services operated a CMH team system, all IPS workers were located within such a team. All IPS workers maintained good or fair levels of IPS fidelity throughout the study (median 65, min–max 61–70 of 75, IPS Fidelity Scale). Vocational Services (N = 156) The vocational service at every centre was chosen on the basis that it was the best alternative VR service available locally, and it was the typical and dominant service in the area. This rehabilitation consisted of an assessment of the patient’s rehabilitation needs, and the provision of a structured training programme aimed at combating deficits related to illness and training in appropriate work skills. The structured programme usually occupied most of the week and was generally at a day centre, although in Ulm it involved mostly residential care.
Outcomes	Percentage of participants who obtained competitive employment Days in competitive employment Hospital admissions Mental health (PANNS, HADS) Quality of life (QOLP) Dropouts
Notes	The primary outcome was the difference in proportions of people entering competitive employment (working for at least 1 day).
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	"Randomisation was done centrally with MINIM (version 1.5)"
Allocation concealment (selection bias)	Low risk	"The allocation sequence was concealed until the services had been assigned"
Blinding of participants and personnel (performance bias)   All outcomes	High risk	Participants and professionals were not blinded to service allocation
Blinding of outcome assessment (detection bias)   All outcomes	High risk	Researchers were not blinded to service allocation
Incomplete outcome data (attrition bias)   All outcomes	High risk	Total of 60 (19%) lost to follow‐up. N = 24 dropped out of IPS group (N = 21 refused interview, N = 3 died). N = 36 dropped‐out of vocational services group (N = 36 dropped out of service of which 27 were still included in study, N = 25 dropped out of study, N = 2 died of natural causes), no reasons provided; N = 2 died). All follow‐up participants receiving IPS were treated; in follow‐up participants in vocational services group, N = 93 were treated and N = 27 were not treated. The study conducted ITT analysis for primary outcome
Selective reporting (reporting bias)	Low risk	All listed outcomes reported
Other bias	Low risk	This study was funded by a grant from the European Union Quality of Life and Management of Living Resources Programme QLRT 2001‐00683. The sponsor of the study had no role in study design, data collection, data analysis, data interpretation, or writing of the report. The corresponding author had full access to all the data in the study and had final responsibility for the decision to submit for publication.

Open in a new tab

Burns 2015.

Methods	Allocation: randomised Design: single centre Duration: 18 months Country: Oxford, UK
Participants	N = 123 Diagnosis: 58% psychotic disorder, ill for an average of 10 years Setting: a nonstatutory mental health service in Oxford (RESTORE) funded by local commissioners and which offered a range of programmes for individuals with mental health problems Age: 18‐65 years, mean 38 years Gender: 59% male Ethnicity: 85% white, 83% born in UK Substance abuse: ‐ Living situation: 89% independent accommodation Marital status: 21% married/cohabiting Employment status: unemployed for a minimum of 6 months, median of 24 months Working history: 97% worked previously Motivation: seeking employment in the open market, referred patients were assessed by the IPS worker for their motivation for obtaining employment before being offered the service Education: median 13 years of general education, 40% entered tertiary education Disability benefit: ‐ Excluded: main reason: duration of unemployment of less than 6 months
Interventions	IPS LITE (N = 62) A shortened form of IPS in which job‐seeking support was limited to 9 months and support to those who acquired employment to 4 months. Those failing to obtain employment were referred back to their mental health team with an open invitation for re‐referral. A similar offer of re‐referral was made to those employed. IPS (N= 61) IPS is a form of VR based on eight principles: (a) focus on competitive employment; (b) no exclusion criteria; (c) rapid job search; (d) integration with mental health team; (e) attention to client’s job preferences; (f) time‐unlimited support; (g) benefits counselling; and (h) active job development. No IPS fidelity measurements reported
Outcomes	Percentage of participants who obtained competitive employment Weeks in competitive employment Days to first competitive employment Mental health (BPRS, HADS) Quality of life (MANSA) Hospital admissions Dropouts
Notes	Definition of competitive employment: employment in the open market for at least 1 day
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	An independent statistician conducted the randomisation. "A non‐deterministic minimisation algorithm was used to produce treatment groups balanced for 5 factors: age, gender, diagnosis, duration of illness and previous employment. The first 2 participants were allocated using simple randomisation to avoid predictability. Subsequently, the minimisation algorithm was applied with an allocation ratio that was not fully deterministic: there was an 80% bias in favour of allocations that minimised the imbalance. If the marginal totals for the groups were the same at a given point, simple randomisation was then used"
Allocation concealment (selection bias)	Unclear risk	No further details
Blinding of participants and personnel (performance bias)   All outcomes	High risk	"Blinding was not possible for the participant, the IPS worker or the independent researcher"
Blinding of outcome assessment (detection bias)   All outcomes	High risk	"Blinding was not possible for the participant, the IPS worker or the independent researcher"
Incomplete outcome data (attrition bias)   All outcomes	High risk	74% follow‐up (N = 48 in intervention group and N = 43 in control group), reasons for missing data described, analyses were conducted according to the ITT principle
Selective reporting (reporting bias)	Low risk	All listed outcomes were reported
Other bias	Low risk	Funding sources: Oxfordshire and Buckinghamshire Mental Health NHS Foundation Trust, National Institute for Health Research (NIHR), Research for Patient Benefit (RfPB) Programme funding (ref: PB‐PG‐0909‐20029)

Open in a new tab

Chandler 1996.

Methods	Allocation: randomised Design: 2 centres, 1 included in this review* Duration: 3 years Country: Long Beach, California, USA
Participants	N = 526 (Long Beach: N = 256) Diagnosis: a serious and persistent mental illness as demonstrated by a DSM III‐R diagnosis, a substantial functional impairment due to the mental disorder and eligible for public assistance as a result of the functional impairment. Included (Long Beach): 55% schizophrenia, 14% bipolar disorder Setting: The ISA (integrated service agency) the Village in Long Beach. The Village management and clinical leadership have wide experience in psychosocial rehabilitation programmes Age: 30% > 45 years Gender: 43% male Ethnicity: 68% white Substance abuse: ‐ Living situation: ‐ Marital status: 47% ever married Employment status: 11%‐12% earned some wages in baseline year (fiscal year before study) Working history: 19% worked at any time in past year Motivation: interest in work was not an eligibility requirement Education: ‐ Disability benefit: 74% received SSI in past year Excluded: primary substance abuse disorder Other: 4% arrested in past year and 2% convicted of a crime past year
Interventions	ISA programme (N = 127) The Village ISA model uses interdisciplinary teams similar to those used in the PACT. The staff‐to‐client ratio is 1:10. Like newer PACT models it integrates services provided by the team with the services of programme specialists in employment, substance abuse and socialisation. There is staff to develop competitive jobs and support clients, where finding employment was a key value of the programme. The Village provided all employment services with its own staff, committing itself to operating a number of businesses at the programme site. These included a cafe and a small store (both open to the public), a catering service, a client bank, and a janitorial service. Besides staffing these transitional employment opportunities, the Village had at least two staff members working to develop competitive jobs and support clients in them. These specialised employment staff and the on‐site job supervisors have worked increasingly closely with the service teams. The integration of transitional employment with supported employment and a core service team is unique. Service as usual (N = 129) Usual mental health services with limited case management and limited amount of other rehabilitative services *Stanislaus ISA The SISA approach varied over time but constant involved outside contract. SISA did not have any in‐house transitional job opportunities nor did it have its own job developers. The core service team rather than specialist staff provided on‐the‐job support.
Outcomes	Percentage of participants who obtained competitive employment Percentage of participants who obtained non‐competitive employment Quality of life (QOLI) Hospital admissions Dropouts
Notes	Competitive employment was not defined but they distinguished completive and transitional employment outcomes *For this review we only used data from one site: village, Long Beach, because we were not able to classify the intervention at SISA in Modesta, and no competitive employment data were available.
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	Clients were randomly assigned
Allocation concealment (selection bias)	Unclear risk	No details
Blinding of participants and personnel (performance bias)   All outcomes	High risk	Participants could identify the given intervention by contents of the programme
Blinding of outcome assessment (detection bias)   All outcomes	High risk	No details about blinding, data collected from statewide data bases and client interviews
Incomplete outcome data (attrition bias)   All outcomes	High risk	Attrition before the study (Long Beach) began reduced the numbers to N = 102 in intervention group and N = 108 in control group (93%). In the 3rd study year service utilisation data were available for N = 95 and N = 86 participants (80%). N = 83 and N = 69 participants were interviewed (67%). In the analyses they used the N = 102 and N = 108
Selective reporting (reporting bias)	Low risk	All listed outcomes are reported
Other bias	Low risk	The study was supported by the California Department of Mental Health and Grant 1R01 MH47063‐03 from the National Institute of Mental Health to Dr Hu

Open in a new tab

Craig 2014.

Methods	Allocation: cluster‐randomised Design: multicentre: 4 Duration: 12 months Country: London and Midlands, UK
Participants	N = 159 Diagnosis: 1st episode psychosis Setting: expected to remain under the care of the early intervention service for at least the subsequent 12 months Age: 18‐35 years, mean 24 years Gender: 73% male Ethnicity: 57% white, 30% black, 14% Asian Substance abuse:‐ Living situation:‐ Marital status:‐ Employment status: not currently in mainstream employment or full‐time education Working history: 73% ever worked, 34% worked since unwell Motivation: all participants had a wish to work Education:‐ Disability benefit:‐ Excluded: organic disease
Interventions	IPS + motivational interviewing (N = 81) Care coordinators were provided with a motivational interview training by recognised experts: 3‐day course, followed by 2 further sessions over the next 3 months and a 2‐day ‘refresher’ course in the second year. The training aimed to provide clinicians with a clear understanding of issues such as intrinsic motivation, ambivalence and readiness to change, as well as how to influence conversations, recognize appropriate times to use motivational interviewing and feel confident in the use of motivational interviewing in everyday practice. Training days consisted of brief didactic work, discussion, role‐play and recorded demonstrations. IPS (N = 78) The vocational specialists in the teams were trained to deliver IPS with fidelity. Scores on this measure for the 4 teams ranged from ‘good’ (both London teams and the intervention team in the Midlands had scores of 111‐114/125) to ‘exemplary’ (the control team in the Midlands with a score of 116). All 4 teams were rated as having IPS fully integrated with the mental health team. However, two teams – one in the intervention arm and one in the control arm – experienced gaps in the availability of IPS specialists when staff moved on or were absent through illness in both instances over comparable 6‐month periods.
Outcomes	Percentage of participants who obtained competitive employment Percentage of participants who obtained non‐competitive employment Weeks in competitive employment Dropouts
Notes	Definition open employment: having a job paying at least minimum wage in a mainstream setting and not specifically for people with disabilities. "A cluster design was chosen to avoid ‘seepage’ from experimental conditions into the control arms"
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	Teams were randomised. in Both clusters 1 team was located in urban setting and 1 team in suburban/rural Design effect
Allocation concealment (selection bias)	High risk	Teams were recruited and baseline motivational training completed before recruitment of participants
Blinding of participants and personnel (performance bias)   All outcomes	High risk	"It was not possible to maintain masking to team allocation"
Blinding of outcome assessment (detection bias)   All outcomes	High risk	Information on occupational activity was obtained from the participants, from the clinical record and in two instances from family source
Incomplete outcome data (attrition bias)   All outcomes	Low risk	No loss of clusters. Occupational outcomes were obtained for 135 participants (85%) at 12‐month follow‐up (N = 68 in intervention group and N = 67 in control group). Sensitivity analyses were carried out.
Selective reporting (reporting bias)	Low risk	All listed outcomes were reported
Other bias	Low risk	The study was funded by a grant from the National Institute for Health Research (NIHR) under its Research for Patient Benefit programme. The views expressed are those of the study authors and not necessarily those of the National Health Service, the NIHR or the Department of Health Significant baseline differences in ethnicity: th the majority of black African and black Caribbean participants in the London teams, the majority of Asian participants in one Midlands team and very few from ethnic minority populations in the second Midlands team Random‐effects logistic regression models used taking account of clustering

Open in a new tab

Dincin 1982.

Methods	Allocation: randomised Design: single centre Duration: 9 months Country: Chicago, Illinois, USA
Participants	N = 132 Diagnosis: severely disturbed people. Included: 75% schizophrenia, 11% other psychosis, 11% major affective disorder,10% personality disorder Setting: Thresholds, a privately operated psychiatric rehabilitation centre. Thresholds offers a full range of programmes to foster improved client functioning in several important areas Age: ≥ 19 years, mean 25.4 years Gender: 53% male Ethnicity: ‐ Substance abuse: ‐ Living situation:‐ Marital status: ‐ Employment status: unclear, but unemployment seems likely because of low (re)employment rates Working history:‐ Motivation:‐ Education: ‐ Disability benefit: ‐ Excluded: not primarily alcohol‐ or drug‐dependent or mentally retarded
Interventions	Comprehensive treatment (N = 66) This treatment consisted of individual casework, VR (i.e. gradual preparation for employment by participation in work crews, and thereafter, in voluntary or paid part‐time positions) and entry‐level jobs in the competitive market place, after having performed adequately in the transitional environments. In addition, social rehabilitation (i.e. problem‐solving and activity groups), linked residential facilities (where suitable), an academic programme (focused on passing the high school equivalency examination), and a medication and relapse discussion group was organised. Incoming clients were assigned to a work crew at Thresholds. After they demonstrated readiness for more demanding tasks they were placed in voluntary or paid part‐time positions in the community. Supportive treatment (N = 66) Supportive treatment programme relied almost exclusively on rehabilitative services and facilities available in the surrounding community. This resulted in referral to existing community services where appropriate. In addition, 6 h/week supportive treatment, which was widely used by practitioners who treat severely disturbed clients, discussion and peer‐support group, and visits fortnightly by a consulting psychiatrist (prescribed and discussed medication) were arranged
Outcomes	Percentage of participants who obtained competitive employment Hospital admissions Dropouts
Notes	Competitive employment, minimum wage was assumed
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	Random assignment at intake
Allocation concealment (selection bias)	Unclear risk	No details
Blinding of participants and personnel (performance bias)   All outcomes	High risk	Participants and personnel could identify given programme by contents
Blinding of outcome assessment (detection bias)   All outcomes	High risk	No details about blinding. Participants were interviewed. Admission data were corroborated by hospital records.
Incomplete outcome data (attrition bias)   All outcomes	High risk	Of the 132 clients who consented to participate (15 in each treatment) were eliminated because they attended fewer than 3 days during the 1st month after intake. At follow‐up they were unable to contact 5 participants of the comprehensive treatment and 14 from the supportive treatment. In the majority of these cases we reconstructed re‐hospitalisation data by talking to reliable informants and hospital records. We were able to obtain verified outcome data for 50 and 43 participants (76%)
Selective reporting (reporting bias)	Low risk	All listed outcomes were reported
Other bias	Low risk	Research was supported by grant 518 from the Illinois department of Mental Health and Developmental Disabilities At the time this work was done Dr. Bond was director of research at Thresholds in Chicago

Open in a new tab

Drake 1996.

Methods	Allocation: randomised Method: multicentre Duration: 18 months Country: New Hampshire, USA
Participants	N = 143 Diagnosis: a major mental illness with major role dysfunction of at least 2 years and clinical stability (i.e. out of the hospital) for at least 1 month 47% schizophrenia and related psychotic disorders, 43% bipolar and other severe mood disorders Setting: CMH centres Age: 20‐65 years, mean 37.0 years Gender: 48% male Ethnicity: 95% white Substance abuse: 20% alcohol use/dependence, 11% drug abuse/dependence Living situation: 84% independent, 9% community residence Marital status: 50% never married, 38% separated/divorced, 10% married, 2.1% widowed Employment status: unemployment for at least 1 month, 36% currently working in non competitive employment (22% sheltered workshop), 0% in competitive employment Motivation: interest in competitive employment Work history: good employment histories Education: 40% > high school, 34% high school, 26% < high school Disability benefit: not reported Excluded: significant memory impairment, medical illness or substance dependence that would preclude participating in a training programme
Interventions	IPS (N = 74) IPS used a team approach to integrate mental health and vocational services. Employment specialists were hired by mental health centres and attached directly to clinical teams to ensure coordinated services. Rather than providing pre‐employment assessment and training in job‐related activities, IPS employment specialists began helping clients to find jobs immediately and, after securing employment, provided training and follow‐along supports as needed. Implementation of IPS differed in the two cities. Both IPS programmes assisted some clients in obtaining volunteer work and sheltered jobs. In one site, these jobs were used as a means of transitioning clients to competitive work. In the second site, however, employment specialists placed more emphasis on sheltered jobs and used them for assessment and long‐term placement, contrary to the IPS model. Despite feedback to supervisors from the project director, this pattern persisted throughout the study, and this site was considered to have a weaker implementation of IPS. The research director monitored implementation through visits and reviewed computerised data. Data generally supported fidelity. Group skills training (GST) (N = 69) The programme offered individualised intake, pre‐employment training in a group format, individualised placement and support on the job, liaison with mental health providers, and follow‐along supports. The pre‐employment training was designed to develop awareness and skills in the three areas of choosing, getting, and keeping a job. In addition to discussing and practicing the skills needed for these tasks, clients were encouraged to explore work‐related values and to understand realistically their strengths and weaknesses as workers. Following the initial skills training, clients met with staff in a group twice each week to continue building interview skills and to discuss potential job leads and interviews. Once employed, clients continued to receive individual support services from GST staff.
Outcomes	Percentage of participants who obtained competitive employment Dropouts Mental health (BPRS, no data available) Quality of life (QOLI, no data available)
Notes	Competitive employment was defined as work in the competitive job market at prevailing wages supervised by personnel employed by the business
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	Clients were stratified on the extent of previous employment and randomly assigned within site
Allocation concealment (selection bias)	Unclear risk	No details
Blinding of participants and personnel (performance bias)   All outcomes	High risk	Participants and personnel could identify the intervention by contents of the programme
Blinding of outcome assessment (detection bias)   All outcomes	High risk	Employment was assessed weekly by employment specialists and by direct interviews with clients. No details about blinding, but unlikely
Incomplete outcome data (attrition bias)   All outcomes	Low risk	Data were obtained for 140 of 143 participants. Two dropped out of the study, and one participant died (IPS lost 2, GST lost 1)
Selective reporting (reporting bias)	Low risk	All listed outcomes reported
Other bias	Low risk	This work was supported by US Public Health Services Grant MH‐00839 from the National Institute of Mental Health and Grant MH‐47650 from the National Institute of Mental Health and the Substance Abuse and Mental Health Services Administration and by the New Hampshire Divisions of Mental Health and Vocational Rehabilitation, the Mental Health Center of Greater Manchester, the Central New Hampshire Community Mental Health Services, and the Employment Connection Specialists. Group equivalence upon study entry was tested for 78 variables. Significant differences were found for two variables. IPS participants were less likely to report that they were not working because of mental disability, and they had more favourable scores on the Rosenberg Self‐Esteem Scale than did GST enrollees

Open in a new tab

Drake 1999b.

Methods	Allocation: randomised Design: single centre Duration: 18 months Country: Washington DC, USA
Participants	N = 152 Diagnosis: criteria SMD of District of Columbia Commission on Mental Health Services: major mental illness, defined as schizophrenia, schizoaffective disorder, bipolar disorder, recurrent major depression or borderline personality disorder, and at least 2 years of major role dysfunction (67% schizophrenia spectrum, 14% bipolar disorder, 16% depressive disorder) Setting: Community Connections, an agency in southeast Washington, DC that serves people with SMD who need intensive case management usually because their psychiatric disorders are complicated by homelessness, comorbid substance use disorder or HIV infection Age: mean 39.4 years Gender: 39% male Ethnicity: 82% African American Substance abuse: 9% current alcohol use disorder, 15% current drug use disorder Living situation: mean 23 days homeless in past year Marital status: 66% never married Employment status: unemployed Motivation: interest in competitive employment Work history: mean 7.7 months paid work in past 5 years Education: 65% high school or higher Disability benefit: not reported Excluded: memory impairment or medical illness that would preclude working or participating in research interviews
Interventions	IPS (N =76) The IPS programme integrated mental health and vocational services by having an employment specialist join multidisciplinary case management teams. IPS employment specialists assisted clients in searching for jobs rapidly and, after securing employment, provided individualised, follow‐along supports as needed without time limits. 3 employment specialist were hired to implement IPS, each had a caseload of 25 clients and carried out all phases of the vocational process. Fidelity rating made regularly throughout the project by the research team indicated that the IPS programme consistently scored within high‐fidelity range of the IPS fidelity scale EVR (Enhanced VR) (N = 76) This approach was considered "enhanced" because an extra vocational counsellor was placed in the Rehabilitation Services Administration office to ensure that participants assigned to this condition were referred to appropriate rehabilitation agencies expeditiously. The vocational counsellor monitored participants monthly, and if a client was dissatisfied with the programme to which he or she was assigned or dropped out of vocational services, the counsellor attempted to link that participant with another agency. All of the EVR agencies endorsed competitive employment as their goal but used stepwise approaches that involved prevocational experiences, primarily paid work adjustment training in a sheltered workshop
Outcomes	Percentage of participants who obtained competitive employment Percentage of participants who obtained non‐competitive employment Days to first competitive employment Quality of life (QOLI) Mental health (BPRS) Dropouts
Notes	Definition of competitive employment: work in competitive job market at prevailing wages with supervision provided by personnel employed by the business and in integrated work settings
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Random assignment was stratified according to work history (> 1 year of employment in a previous job)
Allocation concealment (selection bias)	Unclear risk	No details
Blinding of participants and personnel (performance bias)   All outcomes	High risk	Participants and personnel could identify the given intervention by contents of the programme
Blinding of outcome assessment (detection bias)   All outcomes	High risk	"A limitation was the inability to maintain the blindedness of interviewers"
Incomplete outcome data (attrition bias)   All outcomes	Low risk	2/152 lost to follow‐up, no reasons provided
Selective reporting (reporting bias)	Low risk	All listed outcomes of interest reported
Other bias	Low risk	This work was supported by grant MH51346 from the Substance Abuse and Mental Health Services Administration and the National Institute of Mental Health, Washington, DC, and grant MH00839 from the National Institute of Mental Health

Open in a new tab

Drake 2013.

Methods	Allocation: randomised Design: multicentre Duration: 2 years Country: USA
Participants	N = 2238 Diagnosis: a primary diagnosis of schizophrenia or a mood disorder. Included: 70% affective disorder, 30% schizophrenia Setting: 23 community‐based sites dispersed throughout the USA. All sites had demonstrated the ability to provide high‐fidelity IPS supported employment and integrated behavioural health interventions. Eligible participants were recruited from Social Security Administration lists of SSDI beneficiaries. Age: 18‐55 years, mean 43.5 years Gender: 47% male Ethnicity: 60% white, 26% black, 11% Hispanic Substance abuse: ‐ Living situation: ‐ Marital status: 46% never married Employment status: unemployed Working history: 30% worked in past 2 years Motivation: being interested in gaining employment Education: 12% < high school, 26% high school, 35% some college or technical, 11% bachelor degree Disability benefit: eligibility criteria, 76% on SSDI for > 24 months Excluded: residing in a custodial setting (such as a nursing home), having a legal guardian, having a life‐threatening physical illness that would preclude participating in the study, being currently competitively employed, and already receiving supported employment from the study site
Interventions	Multifaceted intervention (N = 1121) Team‐based SE, systematic medication management, and other behavioural health services, along with elimination of barriers by providing complete health insurance coverage (with no out‐of‐pocket expenses) and suspending disability reviews. The Social Security Administration paid for all of these services and cost‐sharing reimbursements. Other behavioural health interventions were also offered and tailored to participants according to need and preference, e.g. case management, integrated substance abuse treatment, and family psycho‐education. The majority of sites achieved high‐fidelity IPS: 77% in the 1st year, 86% in the 2nd year, and 86% in the 3rd year; 98% of the annual fidelity ratings were fair or high. Usual services (N = 1117) The control group received the same services they had been receiving prior to enrolling. Usual care typically included the services covered by Medicare, such as outpatient physician visits, medications, and hospital care
Outcomes	Percentage of participants who obtained competitive employment Percentage of participants who obtained non/competitive employment (paid employment) Mental health (SFHS) Quality of life (QOLI) Drop outs
Notes	Definition competitive employment: mainstream jobs in integrated work settings at usual wages with regular supervision
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Computer‐generated randomisation at each site
Allocation concealment (selection bias)	Unclear risk	No details
Blinding of participants and personnel (performance bias)   All outcomes	High risk	Participants and personnel could identify given intervention by contents of programme
Blinding of outcome assessment (detection bias)   All outcomes	High risk	No details about blinding. Research interviewers assessed employment status using a computer‐assisted timeline follow‐back calendar
Incomplete outcome data (attrition bias)   All outcomes	Low risk	84.6% followed up, N = 11 in intervention group and N = 13 in control group died and N = 50 and N = 58 < 2 interviews. They used 2 methods to address attrition and missing data. First they considered participants who did not complete at least 2 interviews as non‐responders and adjusted weights to zero. Second, they used imputation procedures to address other participants with missing data
Selective reporting (reporting bias)	Low risk	All listed outcomes were reported, design article published
Other bias	Unclear risk	This study extends work that was conducted under contract SS00‐05‐60072 between the Social Security Administration and Westat. This analysis was conducted for the benefit of the Social Security Administration. The opinions expressed in this article are those of the authors and not necessarily those of the Social Security Administration

Open in a new tab

Drebing 2005.

Methods	Allocation: randomised Design: single centre Duration: 16 weeks Country: Bedford, Massachussetts, USA
Participants	N = 19 Diagnosis: dually diagnosed veterans, defined as those with a current psychiatric diagnosis of schizophrenia, bipolar disorder, major depression, ,PTSD or other anxiety disorder, and current drug or alcohol dependence. Included: 74% affective disorder, 58% anxiety disorder, 11% psychosis Setting: Bedford VA Medical Center Age: mean: 46.0 years Gender: 95% male Ethnicity: 79% white, 21% African American Substance abuse: limited to those with substance dependence for alcohol, cocaine, or opiates with active substance abuse in the prior 90 days. Included: 100% alcohol abuse or dependence, 73% drug abuse or dependence Living situation:‐ Marital status:‐ Employment status: unemployed, mean duration: 4.3 months Working history: potential for return to competitive supported employment within 6 months, as evidenced by a history of participation in competitive employment during the previous 3 years, and acceptance of the stated goal of returning to competitive employment within 8 months Motivation:stated goal of returning to competitive employment Education: mean: 13 years Disability benefit: 32% disability income Excluded: over the age of 55, chronic medical problems that would make it unlikely that they would be able to obtain and sustain a competitive job within 8 months, no intention to stay in the programme for at least 4 months or live in the local region for 12 months, < 10 years of formal education and those with a history of significant head trauma or other disorder resulting in significant cognitive impairment were also excluded
Interventions	CWT only (N = 8) This was a multi‐component work‐for‐pay VR programme. Veterans were placed in structured work settings, usually in private companies, and were compensated for their work. They were typically paid by the CWT programme, which contracts with the company for their labour. While the veterans were working, CWT staff helped them negotiate and resolve difficulties on the job and prepare for obtaining their own competitive jobs. It also included a SE component designed to assist participants in maintaining employment in their own competitive jobs through structured support and management. The Bedford CWT SE services were consistent with published guidelines and treatment fidelity criteria for SE services. CTW + enhanced incentives (N = 13) CWT with enhanced incentives included the benefits available to those who participated in the basic CWT programme, and additional cash awards. Payment of these cash payments or bonuses was contingent on the completion of specified steps leading to obtaining and maintaining employment, maintaining abstinence from substance abuse, and indirectly to more prolonged retention in CWT.
Outcomes	Number of participants who obtained competitive employment Weeks in competitive employment Dropouts
Notes	Pilot study Return to competitive employment, definition was not described The results were not included in the network meta‐analyses and direct comparison meta‐analyses, because we could not classify these interventions in separate groups
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	Participants were randomly assigned
Allocation concealment (selection bias)	Unclear risk	No details
Blinding of participants and personnel (performance bias)   All outcomes	High risk	Participants and personnel could identify the given intervention by contents of the programme
Blinding of outcome assessment (detection bias)   All outcomes	High risk	No details about blinding of outcome assessment but unlikely: data regarding participation and wages were available from CWT clinical and financial records.
Incomplete outcome data (attrition bias)   All outcomes	High risk	Two participants assigned to the payment condition withdrew from participation several days after assignment because of changes in residence, data for these 2 participants were not available for analysis. Only 1 participant, assigned to the incentives condition, failed to complete the 16‐week follow‐up, stopping all participation in Week 15, resulting in no data about competitive employment income for weeks 15 and 16. Missing income and employment data were assumed to reflect no income and no employment. Total lost to follow‐up 3/21 = 14%, but all in intervention groups
Selective reporting (reporting bias)	Low risk	All listed outcomes were reported.
Other bias	Low risk	This study was supported by the New England Mental Illness Research Education and Clinical Center.

Open in a new tab

Drebing 2007.

Methods	Allocation: randomised Design: single centre Duration: 9 months Country: Bedford, Massachussetts, USA
Participants	N = 100 Diagnosis: dually diagnosed veterans, defined as those meeting DSM‐IV criteria for a current diagnosis of (1) schizophrenia, bipolar disorder, major depression, ,PTSD or other anxiety disorder and (2) current drug or alcohol dependence or abuse, as well as active substance use within 90 days of enrolment. They had to be clinically stable, defined as having no suicidal or homicidal ideation in the prior 12 weeks and abstaining from drugs or alcohol for at least 1 week. Setting: Bedford VA Medical Center. Included: 79% major depression, 53% PTSD, 50% other anxiety disorder, 21% bipolar disorder, 9% psychotic disorder. Age: mean: 46.3 years Gender: 99% male Ethnicity: 78% white, 20% African American Substance abuse: all met criteria of dependence of at least 1 substance, 63% poly substance‐dependent. Alcohol 88%, cocaine 43%, cannabis 29%, opiates 26%. Living situation: ‐ Marital status:‐ Employment status: unemployed, mean months unemployed: 16 Working history: potential for return to competitive SE within 6 months, as evidenced by a history of at least some participation in competitive employment during the prior 3 years and acceptance of the stated goal of returning to competitive employment within 8 months Motivation: stated goal of returning to competitive employment Education: mean: 13 years. Disability benefit: 26% disability income, 61% any public support Excluded: older than 55, chronic medical problem that would make obtaining and sustaining a competitive job within 8 months unlikely, no intention to stay in VR for at least 4 months or live in the local region for 12 months, enrolment in other research studies that would affect their participation or difficulty understanding the contingency management programme (< 10 years of formal education, a history of significant head trauma or another disorder resulting in significant cognitive impairment or failed to pass a 10‐item quiz about the incentives)
Interventions	VR (CWT) (N = 50) The CWT is a multi‐component work‐for‐pay VR programme. Veterans were placed in structured work settings, usually in private companies, and were compensated for their work. They were typically paid by the CWT programme, which contracted with the company for their labour. While the veterans were working, CWT staff helped them negotiate and resolve difficulties on the job and prepare for obtaining their own competitive jobs. It also included a SE component designed to assist participants in maintaining employment in their own competitive jobs through structured support and management. The Bedford CWT SE services were consistent with published guidelines and treatment fidelity criteria for SE services. VR (CWT) + contingency management (N = 50) The CWT programme combined with additional incentives for taking steps toward obtaining and maintaining competitive employment and for abstinence from substance use. A series of increasing cash incentives was offered for negative drug and alcohol screens. Employment incentives were available in two phases. Phase I incentives targeted job‐search tasks and were available for the first 16 weeks of the intervention. Phase II incentives targeted employment itself and were available for the first 32 weeks. In total, participants could earn up to USD 610 if they successfully completed all work‐related activities. Over the 36 weeks of the intervention, participants could earn incentives up to USD 1170.
Outcomes	Number of participants who obtained competitive employment
Notes	The current study built on the initial pilot study (Drebing 2005) to determine whether a revised version of this CM intervention applied to a larger sample of VR participants could improve treatment outcomes in terms of the number of participants obtaining and maintaining their own jobs. Competitive employment was defined as the participant working at least 20 h/week in an ongoing community‐based job for which he or she was paid at least minimum wage. The results were not included in the network meta‐analyses and direct comparison meta‐analyses, because we could not classify these interventions in separate groups
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	The participants were randomly assigned
Allocation concealment (selection bias)	Unclear risk	No detail
Blinding of participants and personnel (performance bias)   All outcomes	High risk	Participants and personnel could identify given intervention by content of programme
Blinding of outcome assessment (detection bias)   All outcomes	High risk	"The study is limited by its reliance on self‐report data for key outcome variables"
Incomplete outcome data (attrition bias)   All outcomes	Unclear risk	Two participants, both assigned to the VR‐only condition, dropped out of the study during week 7. The follow‐up rate was 94% for the 3‐month follow‐up, 90% for the 6‐month follow‐up, and 88% for the 9‐month follow‐up. Unclear how many participants dropped out in the intervention or control group. All analyses were based on an ITT approach
Selective reporting (reporting bias)	Low risk	All listed outcomes were reported
Other bias	Low risk	This material was based on work supported by the VA Rehabilitation Research and Development Service (grant D2944R) and with resources of the New England Mental Illness Research, Education, and Clinical Center

Open in a new tab

Eack 2009.

Methods	Allocation: randomised Design: single centre Duration: 2 years Country: Pittsburg, Pennsylvania, USA
Participants	N = 58 Diagnosis: schizophrenia, schizoaffective, or schizophreniform disorder; stabilisation on antipsychotic medications, a time‐span of no greater than 8 years since the onset of first psychotic symptoms and the presence of significant social and cognitive disability, as assessed using the Cognitive Style and Social Cognition Eligibility Interview. Included: 66% schizophrenia, 34% schizoaffective disorder, mean illness duration 3 years Setting: a specialty outpatient clinic in the comprehensive care service at the University of Pittsburgh Western Psychiatric Institute and Clinic, which serves the CMH needs of the majority of schizophrenia population. Age: mean 25.9 years Gender: 69% male Ethnicity: 69% white Substance abuse: ‐ Living situation: ‐ Marital status: ‐ Employment status: 26% employed at baseline Working history: ‐ Motivation: ‐ Education: 67% completed at least some college education, 33% attended college Disability benefit: ‐ Excluded: IQ < 80; significant substance use problems for at least 2 months prior to study enrolment
Interventions	Cognitive enhancement therapy (N = 31) An integrated approach to the remediation of social‐cognitive and neurocognitive deficits in schizophrenia, where participants completed approximately 60 h of computer training in attention, memory, and problem‐solving, and participated in a newly revised, 45‐session, weekly social‐cognitive group that focuses on learning how to take the perspective of others, read non‐verbal cues, manage emotions, and appraise the social context. Enriched supportive therapy (N = 27) An illness management and psycho‐education approach that draws upon components of the basic and intermediate phases of the demonstrably effective personal therapy. Participants were seen on an individual basis. The treatment was divided into 2 phases. Phase I focused on basic psycho‐education about schizophrenia, the role of stress in the disorder, and ways to avoid/minimise stress. Phase II involved a personalised approach to the identification and management of life stressors that pose particular challenges to adequate social and role functioning
Outcomes	Percentage of participants who obtained competitive employment Mental health (BPRS) Dropouts
Notes	Paid competitive employment
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Participants were randomised using a 1:1 ratio by way of computer‐generated random numbers
Allocation concealment (selection bias)	Low risk	"Computer‐generated random numbers"
Blinding of participants and personnel (performance bias)   All outcomes	High risk	Participants and personnel could identify given intervention by content of programme
Blinding of outcome assessment (detection bias)   All outcomes	High risk	Raters were not blind to treatment assignment
Incomplete outcome data (attrition bias)   All outcomes	High risk	N = 67 participants were randomised, 9 excluded, reasons described, unclear intervention allocation of those participants. Analyses were conducted with 58 participants who were randomised and received any exposure, regardless of how limited, to their respective treatment conditions. N = 46 participants completed 2 years of treatment (69% of N =67)
Selective reporting (reporting bias)	Low risk	All listed outcomes were reported
Other bias	Low risk	Supported by NIMH grants MH 79537 and MH 60902. The NIMH had no further role in study design; in the collection, analysis and interpretation of data; in the writing of the report and in the decision to submit the paper for publication.

Open in a new tab

Gervey 1994.

Methods	Allocation: randomised Design: Duration: 1 year Country: New York, USA
Participants	N = 34 Diagnosis: psychiatric disabilities including schizophrenia, major effective disorder, attention deficit disorder, paranoid personality disorder and oppositional defiant disorder Setting: no details Age: 16‐25 years, average 19 years Gender: 67% Ethnicity: 50% African American, 33% Hispanic Substance abuse:‐ Living situation:‐ Marital status:‐ Employment status:‐ Working history: 20% had any work experience Motivation:‐ Education: 80% special education Disability benefit:‐ Excluded:‐
Interventions	SE using job coaching (N = 14) Immediate placement in SE: job placement and job coaching services with weekly individual family and peer group therapy SE using natural support (N = 8) Immediate placement in SE: job placement services with weekly individual, family and peer group therapy Sheltered employment training (N = 12) Employment training in sheltered workshop setting with weekly individual, family and peer group therapy
Outcomes	Percentage of participants who obtained competitive employment Weeks in competitive employment
Notes	We used data of both SE groups as 1 intervention group Before randomisation all participants received vocational and social skills training Competitive employment: minimum wage and at least 20 h/week
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	Randomly assigned
Allocation concealment (selection bias)	Unclear risk	No details
Blinding of participants and personnel (performance bias)   All outcomes	Unclear risk	Participants and personnel could identify the given intervention by contents of the programme
Blinding of outcome assessment (detection bias)   All outcomes	High risk	Data were obtained via records maintained by job developers and interviews. No details about blinding
Incomplete outcome data (attrition bias)   All outcomes	High risk	No details about attrition rate. Preliminary results
Selective reporting (reporting bias)	High risk	No full report published
Other bias	Unclear risk	Funding source unclear

Open in a new tab

Gold 2006.

Methods	Allocation: randomised Design: single centre Duration: 24 months Country: Sumter County, South Carolina, USA
Participants	N = 177 (N = 143 relevant) Diagnosis: meeting both the Federal Center for Mental Health Services’ criteria for severe and persistent mental illness, based upon diagnosis, illness duration, and level of disability 69% schizophrenia spectrum, 31% mood spectrum Setting: The South Carolina Department of Mental Health operated Santee‐Wateree Community Mental Health Center (SWCMHC) Age: ≥ 18 years, 71% between 26‐45 years Gender: 38% male Ethnicity: 77% African American, 19% white Substance abuse: 9% current alcohol abuse/dependence, 8% current drug abuse/dependence Living situation: not reported Marital status: 82.5% not married/not cohabiting Employment status: unemployed Motivation: current and/or future interest in competitive employment Work history: 60% > 6 months paid work in past 5 years Education: 52% high school or higher Disability benefit: 61% Excluded: not reported
Interventions	ACT‐IPS (N = 66) The original study plan was a 3‐group trial comparing 2 newly implemented SE programmes, ACT‐IVR and IPS, to a traditional VR programme. 3 partially implemented and incompletely staffed ACT‐IVR and IPS programmes were integrated into a single ACT‐IPS programme, operating with ACT and IPS subteams composed of the ACT‐IVR and IPS staff members, respectively. To tightly integrate vocational and mental health services, the ACT‐IPS subteams met daily together as a full programme to allocate tasks to each IPS specialist and ACT staff member. Independent ACT and IPS consultants conducted fidelity assessment. IPS Fidelity Scale Yearly averaged total scores rose steadily over the project period, indicating very high IPS model fidelity in years 2–4. ACT fidelity checklist: yearly averaged checklist scores rose steadily over the project period: for years 1–4, indicating high‐fidelity in years 3 and 4. TVR (N = 77) SWCMHC and a local independent nonprofit VR agency specialising in the employment of adults with SMI, signed a formal agreement creating the comparison programme, which they named the Supported Employment Programme (SEP), despite its traditional VR philosophy. SWCMHC provided mental health and brokered case management services in parallel to vocational services. They introduced participants first to graduated work adjustment experiences as preparation for handling competitive job demands. After assessing each participant’s job skills and interests, employment specialists placed participants into 1 of its temporary, staff‐supervised, set‐aside jobs, which differed from competitive jobs in several ways.
Outcomes	Percentage of participants who obtained competitive employment Percentage of participants who obtained non‐competitive employment Weeks in competitive employment Days to first competitive employment Mental health (PANNS) (no data available) Quality of life (QOLI) (no data available) Dropouts Hospital admissions
Notes	Definition competitive employment: the employer (a) paid competitive wages above the federal minimum to participants and workers without disabilities alike, (b) did not set aside the job for adults with disabilities, and (c) located the job in a typical community setting and if (d) the participant contracted for the job Part of Employment Intervention Demonstration Project (EIDP)
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	"SAS‐generated restricted random assignment sequence (permuted blocks of 3)" (SAS is a software programme)
Allocation concealment (selection bias)	Low risk	No investigator was permitted access the assignment sequence
Blinding of participants and personnel (performance bias)   All outcomes	High risk	Participants and personnel could identify the given intervention by contents of programme
Blinding of outcome assessment (detection bias)   All outcomes	High risk	Unblinded study
Incomplete outcome data (attrition bias)   All outcomes	High risk	N = 108 (61%) completed the study. Voluntary withdrawal accounted for most attrition, followed by relocation outside the service area 34 participants were eliminated from the project due to project redesign
Selective reporting (reporting bias)	Low risk	All listed outcomes reported
Other bias	High risk	Project redesign and deviation from a prespecified random assignment process may have compromised study internal validity and programme construct validity. This study was supported by co‐operative agreement SM51823 from the Center for Mental Health Services (CMHS), SAMHSA, and US Department of Health and Human Services (DHHS), Rockville, Maryland, as part of the Employment Intervention Demonstration programme (EIDP), a multi‐site collaboration among 8 research demonstration sites, a co‐ordinating center, and CMHS/SAMHSA. Additional support was provided by grant SM53542 from CMHS/SAMHSA and grant MH01903 from the National Institute of Mental Health (NIMH)

Open in a new tab

Hoffmann 2012.

Methods	Allocation: randomised Design: single centre Duration: 5 years Country: Bern, Switzerland
Participants	N = 100 Diagnosis: stabilised mental disorder in accordance to ICD 10 combined with persistent impairment in the areas of role functioning, social functioning or independent living/self‐care skills. Included: 38% schizophrenia spectrum, 41% affective disorder, mean duration of illness 5.7 years Setting: participants recruited by the Swiss Insurance State Office. Job coach project of the Bern University Hospital of Psychiatry, staffed by employment specialists JCP is part of the CMH division of the Bern University Hospital of Psychiatry Age: 18‐64 years, mean 33.8 years Gender: 65% male Ethnicity: ‐ Substance abuse: 12% concomitant substance abuse Living situation: ‐ Marital status: 74% never married Employment status: out of competitive employment, 18% in sheltered work Working history: mean 24 months of unemployment before intake, employment rate (= ratio between employed/not employed) since age 20: 0.55 Motivation: interest in competitive employment Education: 62% completed vocational training, 25% unskilled or uncompleted, vocational training, 13% university degree Disability benefit: all participants had received the authorisation for VR from the Swiss invalidity insurance Excluded: learning disability (IQ < 70), primary substance abuse disorder, physical or organic handicap that seriously impeded work, unwillingness to attend regular outpatient therapy, performance < 50% of normal work performance as evidenced during the assessment phase; and/or attendance in the programme of < 15 h/week
Interventions	Supported employment (N = 46) The Job Coach Project (JCP) was derived from the IPS model. Some modifications were made in order to meet the standards of the Swiss social insurance system and the needs of the Swiss labour market but also to enhance the sustainability. The JCP was staffed by employment specialists who assisted each participant in the programme in seeking competitive employment. Once employed, on‐the‐job training and follow‐along support was provided. If employment was terminated for any reason, the employment specialist assisted the individual in dealing with job loss and helped to secure another place. The maximum caseload was 12 participants. Several incentives were given to employers. As the JCP was part of the CMH division, the employment specialists were in close contact with the attending therapists from the outset. Repeated IPS fidelity ratings scored between 66 and 68 of 75, i.e. all score sets ranged from equivalent to consistent with the IPS. The sole exception to this was the organisation sub scale, which was not fully consistent, as the law on Swiss Invalidity Insurance requires that an assessment prior to all vocational reintegration programmes be carried out in the form of intake selection. TVR (N = 54) All control interventions had to be verified as high‐quality, train‐place VR programmes and be deemed by the Federal Social Insurance Office to be the best locally available alternative for each prospective participant. Participants in a TVR require a period of preparation before entering into regular employment. They are typically placed in sheltered workshops for 6‐12 months, after which a 3‐6‐month training stint in a companion open market may be feasible. The accompanying support by employment specialists terminated at the end of the TVR.
Outcomes	Percentage of participants who obtained competitive employment Percentage of participants who obtained non‐competitive employment Weeks in competitive employment Days to first competitive employment Mental health (PANNS) Quality of life (W‐QLI) Dropouts Hospital admissions
Notes	Competitive employment was defined as a job on the open labour market that anyone could hold, not only individuals with disabilities. To be counted as competitively employed, the participant had to hold the job for at least 5 days and earn at least a minimum wage. All fidelity ratings were performed by the research team We used data after 2 years for this review (to be comparable to other study follow‐up data)
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	"Randomisation was performed based on a random list generated by a computer algorithm. This list was randomly chosen from a choice of 10 lists and was then transformed in a stack of sequentially numbered and sealed envelopes containing individual assignments. These steps were performed by an administrative office outside the research team."
Allocation concealment (selection bias)	Low risk	"The randomisation procedure guaranteed that the research team was always fully blinded regarding to assignment." Sealed envelopes were used.
Blinding of participants and personnel (performance bias)   All outcomes	High risk	"Group assignment was only revealed once all initial assignments had been completed."
Blinding of outcome assessment (detection bias)   All outcomes	High risk	"Interviewer was not blind to programme assignments, thereby giving rise to possible rater bias. However, key employment measures were objective and duly corroborated by multiple sources."
Incomplete outcome data (attrition bias)   All outcomes	Low risk	Lost to follow‐up: 7/100. Missing outcome data balanced in numbers across intervention groups, with similar reasons for missing data across groups. ITT analysis performed
Selective reporting (reporting bias)	Low risk	All listed outcomes were reported
Other bias	Unclear risk	Swiss National Science Foundation, Grand 3200‐064032, unconditioned grants of the Federal Social Insurance Office, the Stanley Thomas Johnson Foundation, the Gottfired and Julia Bangerter‐Rhyner Foundation, the Bank Vontobel Foundation, the Dosenbach‐Waser Foundation and the Karl Mayer Foundation. Funder is also recruiter of participants and the income resource

Open in a new tab

Howard 2010.

Methods	Allocation: randomised Design: multicentre Duration: 24 months Country: London, UK
Participants	N = 219 Diagnosis: severe mental illness: duration of illness over 2 years, global assessment of functioning (GAF) 60 or less and a SCAN diagnosis of a psychotic or chronic affective disorder 73% psychotic disorder, 27% mood disorder Setting: CMH teams in 2 boroughs of South London Age: between 18‐65 years, mean 38.5 years Gender: 67% male Ethnicity: 43% black, 38% white Substance abuse: not reported Living situation: 54% living alone Marital status: not reported Employment status: unemployed for at least 3 months Work history: 54% worked in the past 5 years Motivation: wanting to obtain competitive employment Education: not reported Disablity benefit: not reported Excluded: IPS in the previous 6 months
Interventions	IPS (N = 109) 4 experienced employment specialists were linked with CMH teams. They focused on rapid placement with continued follow‐up support and sought to find employment opportunities that were consistent with participants’ preferences, skills and abilities. However, the integration of the IPS programme was not structural or managerial. IPS fidelity was found to be high; the IPS programme in the two boroughs received a good IPS rating; the IPS programme scored less well on the organisation dimension (specifically, integration of rehabilitation with mental health treatment) and, for Borough A, the services dimension (specifically, for the rapid search for a competitive job) compared with other dimensions. Treatment as usual ( N = 110) Treatment as usual consisted of existing psychosocial rehabilitation and day care programmes available in the local area. A range of courses were offered, most commonly pre‐employment preparation (e.g. interview skills, curriculum vitae coaching and application form practice), computers/information technology and confidence building/motivation
Outcomes	Percentage of participants in competitive employment Mental Health (BPRS) Quality of life (MANSA, QOLP) Hospital admissions Dropouts
Notes	Definition of competitive employment: a job paying at least the minimum wage, located in a mainstream socially integrated setting not set aside for persons with disabilities, held independently (i.e. not agency owned), with the participant in continuous employment for at least 30 days
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	"Treatment allocation was stratified by gender and age (10‐year bands). Randomisation with minimisation was used."
Allocation concealment (selection bias)	Low risk	Randomisation with minimisation was used, performed by the Institute of Psychiatry Mental Health and Neuroscience Clinical Trials Unit, a unit independent of the study to maintain concealment.
Blinding of participants and personnel (performance bias)   All outcomes	High risk	"It was not possible for participants or those administering the intervention to be masked to the participants’ allocation status"
Blinding of outcome assessment (detection bias)   All outcomes	High risk	"The researchers who conducted the 1‐year follow‐up interviews were masked to allocation status, but guessed 119 correctly out of the 197 (60%) clients assessed compared with a hypothesised 50% (with random guesses); this is significant at P = 0.005."
Incomplete outcome data (attrition bias)   All outcomes	Low risk	N = 15 (14%) in each group lost to follow‐up after 2 years: 20 withdrawn, 2 out of the country, 2 missing, 2 too ill to follow up, 3 unable to find. Furthermore, In the intervention group 1 participant was excluded from analysis because the researchers were unable to ascertain job status. No significant differences in sociodemographic or clinical variables between those who were and those who were not lost to follow‐up. All data were analysed in groups as randomised, whether or not receiving an intervention (i.e. ITT).
Selective reporting (reporting bias)	Low risk	All listed outcomes reported
Other bias	Low risk	This study was supported by the Wellcome Trust (GR071272MA); the supported employment programme was funded partly by the King’s Fund and the South London and Maudsley Charitable Trust

Open in a new tab

Killackey 2008.

Methods	Allocation: randomised Design: single centre Duration: 6 months Country: Melbourne, Australia
Participants	N = 41 Diagnosis: first episode psychosis. All participants received clinical diagnoses of schizophrenia‐spectrum disorders. Mean (SD) length of illness: intervention 12.25 (12.98) and control 15.68 (14.17) Setting: Early Psychosis Prevention and Intervention Centre (EPPIC) Age: between 15‐25 years, mean 21.4 years Gender: 80% male Ethnicity: ‐ Substance abuse: N = 23 had substance use diagnoses Living situation: 61% lived with family of origin, 36% rented a house/flat Marital status: 81% never married Employment status: 8% employed at baseline (N = 1 in intervention group and N = 2 in control group had a job) Work history: N = 34 reported a work history Motivation: wanted to find work, including a different job if they currently held one Education: 44% completed trade/technical training, 28% completed secondary education or partially completed tertiary training Disability benefit: 78% welfare benefits Excluded: lack of fluency in English Other: 92% absence of medical illness
Interventions	IPS (N = 20) IPS is comprised of seven key principles: (1) a focus on competitive employment; (2) open to any person with a mental illness; (3) utilises a rapid job search approach; (4) integrated with mental health treatment team; (5) potential jobs are chosen based on people’s preferences; (6) time unlimited and provides individualised support; and (7) welfare benefits counselling is provided as monetary disincentives often need to be negotiated in the transition from a welfare benefit to paid employment. Additionally, the IPS model is extended to integrate supported education, given that vocational goals of this population frequently include completion of schooling or further training prior to job placement. An employment consultant was employed to deliver the vocational intervention which was carried out with high‐fidelity according to the Supported Employment Fidelity Scale. The fidelity was assessed by the project leader and reviewed with an interstate colleague independent of the project. Treatment as usual (N = 21) Treatment as usual consisted of participants continuing to receive EPPIC care. This involves individual case management and medical review, referral to external vocational agencies, as well as involvement with the group programme at EPPIC, which may involve participation in the vocationally oriented groups within the group programme. Treatment as usual was delivered primarily by EPPIC case managers.
Outcomes	Percentage of participants who obtained competitive employment Number of weeks in competitive employment Drop outs
Notes	Definition competitive employment: jobs which are not set aside but open to applications from anyone with the appropriate skills or qualifications
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	"Participants were randomised by a statistician independent of the study using computer‐generated random numbers to carry out blocked randomisation to one of the two conditions"
Allocation concealment (selection bias)	Low risk	"The statistician was contacted by the leader of the project when a new participant enrolled and the statistician informed the leader of the group allocation. This information would then be given to the participant, the case manager of the participant and also the employment consultant if allocation was to the intervention group."
Blinding of participants and personnel (performance bias)   All outcomes	High risk	Information about allocation was given to leader of the project, participant, case manager and employment consultant
Blinding of outcome assessment (detection bias)   All outcomes	High risk	Assessments were conducted by a research assistant. The research assistant was not informed about allocation, but there were no formal tests of her masking to allocation.
Incomplete outcome data (attrition bias)   All outcomes	Low risk	1 participant dropped out from the intervention group and 5 from the control group. In the control group 4 dropped out because they had enrolled wanting help to find work and felt that as they were not getting it they no longer wished to continue in the project. The remaining 2 participants dropped out as they were sent to jail. However, all who dropped out gave their permission for their employment status at follow‐up
Selective reporting (reporting bias)	Low risk	All listed outcomes reported
Other bias	High risk	This research was supported by a National Health and Medical Research Council programme Grant (ID: 350241) and an unrestricted study grant from Bristol Myers Squibb. Orygen Research Centre is supported by the Colonial Foundation There was a significant baseline difference in marital status. As more people in the control were in marital or marital‐like relationships. In logistic regression analysis there was a 6.65 more chance to have worked if married (P = 0.044)

Open in a new tab

Killackey 2014.

Methods	Allocation: randomised Design: single centre Duration: 18 months Country: Melbourne, Australia
Participants	N = 146 Diagnosis: first episode psychosis Setting: Early Psychosis Prevention and Intervention Centre (EPPIC), which is a sub‐programme of Orygen Youth Health. Orygen Youth Health is a public mental health service for young people aged 15–25 years Age: 15‐25 years, mean 20.3 years Gender: 67% male Ethnicity: 75% country of birth was Australia Substance abuse: not reported Living situation: not reported Marital status: 97% never married Employment status: 16% employed at baseline Working history: not reported Motivation: expressed an interest in employment or education, whether that was to find a job or course or to receive support to stay in a current job or course Education: 42% completed year 12, 18% year 11 and 18% year 10, 22% year 7‐9 Disability benefit: not reported Excluded: having severe intellectual disability or having florid psychosis that prevented the determination of ability to provide informed consent. Lack of fluency in English
Interventions	IPS (N = 73) In addition to receiving treatment as usual, those in the IPS group received service from an employment consultant working according to the IPS model located on site at EPPIC. The employment consultant's job was to meet with clients as soon as possible after randomisation and provide them with an employment service based on the eight principles of the IPS model described above. No details about fidelity, but this study was conducted at the same centre as Killackey 2008 with high‐fidelity scores. Treatment as usual (N =73) Those in the TAU group received all the services that they were eligible for as clients of EPPIC. This included medical review, clinical case management, group programmes (some of which were vocationally oriented) and referral by case managers to offsite agencies (e.g. housing, welfare or employment agencies). In addition, a local Department of Employment and Workplace Relations contracted employment service had an employment consultant on site at Orygen for one half‐day per week, to whom all clients were able to be referred by case managers if this was deemed appropriate
Outcomes	Percentage of participants who obtained competitive employment (only preliminary data after 6 months follow‐up available) Dropouts
Notes	No details about definition of competitive employment Outcome data derived from a systematic review (Bond 2015a)
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	"The participants were allocated using a computer programme for blocked randomisation in random permutated blocks of 4 and 8"
Allocation concealment (selection bias)	Low risk	"Randomisation was conducted by the study statistician who was not associated with assessments and treatments. She was the only person aware of the allocation sequence. She provided the group allocation to the study lead who informed the employment consultant and the participant’s case manager of the group to which the participant had been allocated"
Blinding of participants and personnel (performance bias)   All outcomes	High risk	The employment consultant and the participant’s case manager were informed about group allocation. Participants could also identify allocation by contents of the programme
Blinding of outcome assessment (detection bias)   All outcomes	Low risk	"All effort was taken to keep research assistants (RAs) blinded to study condition. RAs had no contact with the employment consultant, and participants were reminded at the start of each assessment that they were not to let the RA know whether they had been working with the employment consultant or not."
Incomplete outcome data (attrition bias)   All outcomes	High risk	86% follow‐up
Selective reporting (reporting bias)	Unclear risk	Only preliminary data were available. Not all outcomes as listed in design article were reported
Other bias	Low risk	Supported by Australian Rotary Health; the Australian Research Council (LP0883273); Orygen Youth Health Research Centre; a National Health and Medical Research Council Clinical Research Fellowship (#628884); and University of Melbourne, Faculty of Medicine, Dentistry and Health Sciences Ronald Philip Griffiths Fellowships

Open in a new tab

Latimer 2006.

Methods	Allocation: randomised Design: single centre Duration: 12 months Country: Montreal, Canada
Participants	N = 150 Diagnosis: schizophrenia spectrum disorder (schizophrenia, schizoaffective disorder, schizophreniform disorder, psychosis not otherwise specified), bipolar disorder, or major depression. If their principal diagnosis was one of major depression, were classified as disabled due to mental illness by the provincial welfare system. Included: 59% schizophrenia spectrum and 20% bipolar disorder Setting: Douglas Hospital, a teaching psychiatric hospital with a VR centre Age: 18‐64 years, mean 40.3 years Gender: 62% male Ethnicity: 82% white Substance abuse: 4% alcohol misuse or dependence, 10% drug misuse or dependence Living situation: 22% residential services Marital status: 79% single/never married Employment status: unemployed Working history: 31% competitive employment in past 5 years, 42% paid non‐competitive work in past 5 years Motivation: express interest in competitive employment Education: 43% > 12 years Disability benefit: not reported Excluded: learning disability (IQ < 70), physical or organic handicap that seriously impeded work or did not have a case manager willing to see them at least once per month
Interventions	Supported employment (N = 75) SE specialists helped the clients to (a) define a competitive job corresponding to his or her interests and capabilities; (b) obtain such a job; (c) continue in employment, once a job was obtained; (d) recover from job loss, identifying what went wrong and looking for a new employment opportunity. Fidelity was assessed on 2 occasions using the Supported Employment Fidelity Scale 11 months and 2.5 years after the programme was initiated. The two ratings were consensus ratings between two different pairs of investigators. Both ratings indicated good implementation of IPS. Usual vocational services (N = 75) Clients were given an opportunity to sign up for one of the many vocational services normally available. These included sheltered workshops, creative workshops, a client‐run boutique and horticultural programmes. Job‐finding skills training, as well as psychosocial interventions administered through two day‐treatment centres, were also available. None of these programmes had competitive employment as their immediate goal. In addition, clients could be offered a social integration measure, that is a Québec government programme that offers clients part‐time work in competitive settings, in exchange for a CAD 120 top‐up to their monthly welfare cheque and a free public transport pass. Finally, clients could also be referred to a non‐profit community agency that sought to place clients either in competitive jobs or in government subsidised adapted businesses, in which wages equal or exceed the legal minimum wage but where the majority of jobs are reserved for people who have disabilities. This agency was not integrated with clinical services, nor did it provide ongoing support to clients.
Outcomes	Percentage of participants who obtained competitive employment Percentage of participants who obtained non‐competitive employment ("any paid jobs") Weeks in competitive employment Days to first competitive employment Dropouts
Notes	Jobs were categorised as competitive if: they paid the minimum wage or better, or on a commission basis (e.g. sales); they were not reserved for people with disabilities; and fewer than 50% of the person’s coworkers had disabilities (information ascertained by contacting the employer directly)
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	"Stratified randomisation was used, with two factors that were expected to influence vocational outcomes: previous work history (≥ 1 year of continuous work experience at some point in the past, or less) and clinical site. The biostatistician associated with the study generated 16 random assignment sequences"
Allocation concealment (selection bias)	Low risk	"The project co‐ordinator prepared an opaque envelope containing the assignment and gave it to the interviewer before the baseline interview"
Blinding of participants and personnel (performance bias)   All outcomes	High risk	"Assignment was revealed to both interviewer and participant at the conclusion of the baseline interview"
Blinding of outcome assessment (detection bias)   All outcomes	High risk	Interviewers were not masked to group assignment
Incomplete outcome data (attrition bias)   All outcomes	Low risk	17% lost to follow‐up, N = 9 in control group and N = 16 in intervention group. Reasons seem to be similar in both groups. In total 5 moved, 17 refused, 3 had other reasons. ITT analysis used
Selective reporting (reporting bias)	Low risk	All listed outcomes were reported
Other bias	Low risk	This study was funded by the Canadian Institutes of Health Research, the Quebec Health Research Fund and AETMIS

Open in a new tab

Lecomte 2014.

Methods	Allocation: randomised Design: single centre Duration: 12 months Country: Montreal, Québec, Canada
Participants	N = 24 (total sample 160) Diagnosis: severe mental illness. Included: 63% schizophrenia, 33% mood disorder Setting: participants enrolled in supported employment programmes in Montreal Age: mean 32.4 years Gender: 71% male Ethnicity: ‐ Substance abuse: ‐ Living situation: ‐ Marital status: ‐ Employment status: ‐ Working history: ‐ Motivation: wish to obtain a competitive job Education: 46% secondary education, 33% university, 17% Baccalauréat (academic qualification for entrance into university) Disability benefit: ‐ Excluded: zero exclusion
Interventions	SE + CBT (N = 12) Group CBT intervention offered during 8 sessions over the course of 1 month, in order to respect the rapid job search principle of IPS, was developed. The content was tailored to facilitate the learning of skills specific to the work‐place, such as recognising and managing one’s stressors at work, determining and modifying dysfunctional thoughts,overcoming obstacles, improving one’s self‐esteem as a worker, dealing with criticism, using positive assertiveness, finding coping strategies to use at work, negotiating work accommodations and overcoming stigma. SE (N = 12) A SE following the 8 principles of IPS No fidelity measurements reported
Outcomes	Number of participants who obtained competitive employment Number of participants who obtained non‐competitive employment Weeks in competitive employment
Notes	Preliminary results of a subsample Definition of competitive employment: not reserved for people with disabilities, at least minimum wages and full time or part time
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	"Un essai randomisé contrôlé" (a randomised controlled trial)
Allocation concealment (selection bias)	Low risk	No details in article, low according to study author's information
Blinding of participants and personnel (performance bias)   All outcomes	High risk	Participants and personnel could identify the given intervention by contents of the programme
Blinding of outcome assessment (detection bias)   All outcomes	High risk	Unclear if interviewers were blinded
Incomplete outcome data (attrition bias)   All outcomes	High risk	Preliminary results of a sub sample, no dropouts reported
Selective reporting (reporting bias)	Unclear risk	There was a slight bias in reporting since the preliminary analyses only focused on some of the measures, not all were analysed
Other bias	Low risk	No details about funding source, but they reported no conflicts of interest

Open in a new tab

Lehman 2002.

Methods	Allocation: randomised Design: single centre Duration: 24 months Country: Baltimore, Maryland, USA
Participants	N = 219 Diagnosis: criteria of severe mental illness based on diagnosis, duration of illness and level of disability. Patients were automatically eligible if they received SSI, SSDI or VA disability benefits because of a mental disorder (other than substance abuse only) or if they had a schizophrenia spectrum disorder using DSM IV criteria. Those not meeting these criteria were eligible if they had another axis I disorder (other psychotic, major affective or anxiety disorder) or an extensive prior hospitalisation history. Finally patients were included if they had had a history of a mental disorder for at least the past year, during which they were unable to spend 75% of their time in some gainful activity owing to the mental disorder. Included: 75% psychotic disorder, 25% mood disorders Setting: outpatient psychiatric care from 3 continuous care teams within a university‐run CMH agency serving inner‐city Baltimore Age: mean 41.5 years Gender: 57% male Ethnicity: 75% African American or other minority Substance abuse: 40% substance use diagnosis with current use, 50% with use in past 5 years, 75% with lifetime use Living situation: 52% independent, 24% supported or assisted, 18% with family, 6% homeless Marital status: 62% never married, 34% divorced/separated/widowed Employment status: at least 3 months unemployed Working history: 48% at least 1 job in past 5 years Motivation: not reported Education: 51% high school graduate or GED, 49% did not complete high school Disability benefit: 89% SSDI, SSI or combination Excluded: not reported
Interventions	IPS (N =113) This model focuses on a rapid job search with continued follow‐along support. The IPS programme seeks employment opportunities that are consistent with participants’ preferences, skills, and abilities. Fidelity ratings, completed by the IPS programme developer who served as a consultant to the project, were made twice yearly using the IPS Fidelity Scale. The programme received high ratings of implementation fidelity across all review periods. Psychosocial rehabilitation (N =106) The comparison programme provided an array of services, including evaluation and skills training, socialisation, access to entitlements, transportation, housing supports, counselling and education. Vocational services included in‐house evaluation and training for individuals who staff believed were not yet fully prepared for competitive employment. Training focused on improving specific work readiness skills, such as work endurance, appropriate social interaction in the workplace, and acceptance of supervision. In‐house sheltered work and factory enclave projects were also available. For those ready for competitive employment, the psychosocial programme either provided in‐house assistance in securing employment or referred participants to city‐based rehabilitation or vocational service programmes
Outcomes	Percentage of participants who obtained competitive employment Percentage of participants who obtained non‐competitive employment Dropouts
Notes	Definition competitive employment: a job in which the worker earned at least minimum wage and the worker had no contact with disabled workers and at least some contact with non‐disabled workers, and the job had not been set aside for a disabled person. Part of Employment Intervention Demonstration Project (EIDP)
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	"Using pre‐prepared sealed envelopes participants were randomly assigned." No further details
Allocation concealment (selection bias)	Low risk	Randomised using pre‐prepared sealed envelopes
Blinding of participants and personnel (performance bias)   All outcomes	High risk	Participants and personnel could identify the given intervention by contents of the programme
Blinding of outcome assessment (detection bias)   All outcomes	High risk	Employment data (primary outcome) were collected by using an employment form completed by case managers or vocational specialists
Incomplete outcome data (attrition bias)   All outcomes	High risk	N = 148 completed study, 26% lost to follow‐up in intervention group and 60% in control group
Selective reporting (reporting bias)	High risk	Not all listed outcomes were reported (e.g. quality of life, self‐esteem at follow‐up)
Other bias	Low risk	This study was supported by co‐operative grant UD7‐ SM51824 from the Center for Mental Health Services, Substance Abuse and Mental Health Services Administration, Department of Health and Human Services, Rockville, Md, as part of the Employment Intervention Demonstration Project; grant P50‐MH4370 from the National Institute of Mental Health, Rockville; and the Mental Illness Research Education and Clinical Center, Veterans Affairs Integrated Service Network 5, Baltimore. "The contents of this publication are solely the responsibility of the authors and do not necessarily represent the official views of the centre for mental health services, the substance abuse and mental health services administration, the department of health and human services or other employment intervention demonstration project collaborators" Participants received USD 20 for baseline interviews, USD 10 for each of next 2 follow‐up interviews and USD 15 for the 18‐ and 24‐month interviews

Open in a new tab

McFarlane 1996.

Methods	Allocation: randomised Design: multi centre Duration: 24 months Country: New York State, USA
Participants	N = 68 Diagnosis: DSM‐III‐R criteria for schizophrenia, schizoaffective disorder or schizophreniform disorder with also one or more additional complicating factors. Setting: the study was conducted at 3 mental health centres, 1 each in an urban, suburban, and rural area of New York State. Participants were selected during an admission to an inpatient service or an acute partial hospital or when they were receiving crisis services for an acute psychotic episode. In total 28% of the participants was in hospital when they entered the study. Age: 18‐45 years, mean 29.8 years Gender: 65% male Ethnicity: 78% white Substance abuse: 80% no history of abuse Living situation: 47% lives with family, 25% supervised without family, 26% unsupervised without family, Marital status: 84%never married, 9% divorced/separated Employment status: 91.2% unemployed, 5.9% sheltered work, 2.9% competitive employed Working history:‐ Motivation:‐ Education: 30% high school graduate, 29% some college, 22% some high school Disability benefit: ‐ Excluded: acutely violent or suicidal people and those with major medical illness or physical addiction requiring immediate medical hospitalisation were excluded until they were stabilised or detoxified.
Interventions	ACT + multifamily group (N = 37) After an initial psycho‐educational workshop for family members only, multifamily groups, each comprising 6 participants and their families, met with two ACT team members every other week for 2 years. The teams were guided by participants’ and family members’ preferences and intentions. One modification was made in the ACT approach: a more gradually paced recovery and rehabilitation concept was adopted from family psychoeducation to ensure the lowest risk of relapse and more consistent development of independent living skills ACT + crisis family intervention (N = 31) Co‐ordination between the team and family members occurred only during crises, without the input of other participants’ family members
Outcomes	Number of participants who obtained competitive employment Number of participants who obtained non‐competitive employment Mental health (PANSS)
Notes	We could not classify this comparison for the network meta‐analyses and direct comparison meta‐analyses No definition of competitive employment described, but they made a distinction between sheltered and competitive employment
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	Participants and their families were randomly assigned
Allocation concealment (selection bias)	Unclear risk	"Post‐treatment interviews with the project staff failed to detect any bias in assigning participants to treatment conditions"
Blinding of participants and personnel (performance bias)   All outcomes	High risk	Personnel and participants were not blinded. They could identify assignment by contents of programme
Blinding of outcome assessment (detection bias)   All outcomes	Low risk	"Raters were independent and blind to the treatment condition"
Incomplete outcome data (attrition bias)   All outcomes	Low risk	No differences in attrition: three cases in each cohort (6/68)
Selective reporting (reporting bias)	Low risk	All listed outcomes were reported
Other bias	High risk	New York State Alliance for the Mentally Ill, which sponsored the project, and the New York State Office of Mental Health, which provided funding Baseline differences were considered chance effects and were entered as control variables in subsequent analyses. Participants in the intervention condition received care from the same treatment teams that provided services to participants in the control condition. The possibility of clinician bias existed.

Open in a new tab

McFarlane 2000.

Methods	Allocation: randomised Design: multi centre Duration: 18 months Country: New York City, New York, USA
Participants	N = 69 Diagnosis: a diagnosis in either the schizophrenia or the mood disorder spectrum. Included: 73% schizophrenia spectrum, 27% mood spectrum Setting: 2 New York state CMH centres , 1 of the 2 service sites was located in an increasingly urbanised suburb of New York City (New Rochelle, Westchester County) and the other in rural New York state (Kingston, Ulster County) Age: 18‐55 years, mean 33.0 years Gender: 70% male Ethnicity: 87% white Substance abuse: 91% none/rare alcohol use, 94% non/rare other substance use Living situation: 36% lived with family member, 64% non family Marital status: 74% never married, 13% separated/divorced Employment status: not employed competitively for the past 6 months Working history: mean months since last job 15, mean number of prior jobs: 6 Motivation: explicit wish to work Education: 34% high school graduate, 21% not high school graduate, 35% some college, 10% college grad Disability benefit: 64% SSI, 49% SSD, 4% SSA Excluded:‐
Interventions	FACT (N = 37) Family‐aided Assertive Community Treatment consisted of ACT, family intervention and vocational specialists. The vocational specialists were trained by Becker (one of the founders of IPS). Their specific tasks were to: I ) lead 9‐session goal‐setting groups; 2) work with each individual to identify and contact potential employers; 3) work on job development for the entire cohort, to find co‐operative potential employers; 4) coach participants on and off the job site in the initial month or two of employment; 5) provide technical assistance to their team‐mates in job‐coaching; and 6) develop methods for assessing work‐readiness, preparing resumes, and practicing interviewing skills TVR (N = 32) TVR with referral to state VR service often leading to placement in sheltered workshop. Case loads were heavier
Outcomes	Number of participants who obtained competitive employment Weeks in competitive employment Number of participants who obtained non‐competitive employment
Notes	No definition of competitive employment described, but they made a difference between type of employment (sheltered employment, supported employment, vocational training, competitive employment) No IPS fidelity measurements reported Part of Employment Intervention Demonstration Project (EIDP)
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	Participants were randomly assigned
Allocation concealment (selection bias)	Unclear risk	No details
Blinding of participants and personnel (performance bias)   All outcomes	High risk	Participants and personnel could identify the given intervention by contents of the programme
Blinding of outcome assessment (detection bias)   All outcomes	High risk	Clinicians completed employment trackings form for each subject
Incomplete outcome data (attrition bias)   All outcomes	Unclear risk	No details about attrition rate
Selective reporting (reporting bias)	Low risk	All listed outcomes were reported
Other bias	Low risk	Supported by grant R18 SM 47642 from the National Institute of Mental Health

Open in a new tab

McGurk 2007.

Methods	Allocation: randomised Design: multi centre Duration: 2‐3 years, average follow‐up was 26 months Country: New York City, New York, USA
Participants	N = 48 Diagnosis: severe mental illness as determined by the State of New York Office of Mental Health. Included: 73% schizophrenia, 4% schizoaffective disorder, 23% mood disorder Setting: 2 CMH centres. Both agencies serve predominantly people from minority groups and provide a comprehensive range of services: housing, psychiatric, community support, day treatment, psychosocial, and integrated vocational services (including SE) Age: mean 37.6 years Gender: 55% male Ethnicity: 68% African American, 16% Hispanic Substance abuse: 9% current alcohol use, 30% lifetime alcohol use, 7% current drug use, 53% lifetime drug use Living situation: ‐ Marital status: 86% never married Employment status: not currently competitively employed Working history: all participants had at least one unsatisfactory job ending. Mean time since last job: 40 months Motivation: desire for competitive work Education: mean: 11 years of education Disability benefit:‐ Excluded:‐
Interventions	SE + cognitive training (N = 25) The Thinking Skills for Work Program was designed as an adjunct to SE. It was divided into 4 component parts and was delivered by a cognitive training specialist. Clients were engaged in 24 hours of computer‐based cognitive exercises (Cogpack v6) providing practice across the broad range of cognitive functions, including attention and concentration, psychomotor speed, learning and memory, and executive functions. Sessions required 45–60 minutes to complete, with clients usually completing 2–3 sessions per week for a total duration of about 12 weeks. Participants received performance scores in order to reinforce performance progress. The cognitive training specialist, the employment specialist, and the client met together to plan the job search, based on the client’s preferences. Then, a meeting was held to review the client’s job interests, to evaluate his or her cognitive strengths and gains made in the computer cognitive training exercises, and to consider possible support to compensate for cognitive impairments that could compromise work performance. The cognitive training specialist and the employment specialist met regularly to discuss job supports. Supported employment (N = 23) Both sites had SE programmes that broadly adhered to evidence‐based definitions of the practice. Both programmes had zero exclusion criteria, no PVT, minimal prevocational assessment, emphasis on rapid job search for competitive employment in integrated community settings, attention to consumer preferences with respect to jobs sought and disclosure of psychiatric disability, and provision of follow‐along supports to facilitate job retention Fidelity was rated with the Supported Employment Fidelity scale. The first 2 study authors conducted the fidelity assessments with sites receiving ratings of "fair" (score 60) and "good" (score 66) implementation
Outcomes	Number of participants who obtained competitive employment Weeks in competitive employment Mental health (PANSS) Hospital admissions (no usable data) Dropouts
Notes	Competitive work was defined as jobs paying minimum wage or higher, owned by the individual, not set aside for a person with a disability, and integrated in the community
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	"Assignment to the programmes was conducted on the basis of a computer‐generated randomisation list"
Allocation concealment (selection bias)	Unclear risk	"Individuals were randomly assigned within each centre." No further details
Blinding of participants and personnel (performance bias)   All outcomes	High risk	Participants and personnel could identify given intervention by content of programme
Blinding of outcome assessment (detection bias)   All outcomes	High risk	Vocational outcomes were assessed through regular contacts with clients and vocational staff members. No details about blinding but unlikely
Incomplete outcome data (attrition bias)   All outcomes	High risk	48 clients were randomised (25 in intervention and 23 in control group). Soon after randomisation 2 clients (1 in intervention group, 1 in control group) withdrew consent from the study due to serious medical conditions, and 1 client (control) died. One client left vocational services and was lost to follow‐up within a month after randomisation. Twelve‐month employment data were available on 32 (67%) clients, rates of follow‐up were comparable between the 2 groups. No details about attrition rate after 2‐3 years
Selective reporting (reporting bias)	Low risk	All listed outcomes are reported
Other bias	Low risk	Supported by grant from the Center for Rehabilitation and Recovery, Coalition of Behavioral Health Agencies, Inc., New York Clients recruited for the project at CMHC A differed in a number of background and baseline characteristics from clients recruited from CMHC B. Because of these site differences, they included site as an independent variable. No significant differences between intervention and control group

Open in a new tab

McGurk 2009.

Methods	Allocation: randomised Design: single centre Duration: 24 months Country: New York, New York, USA
Participants	N = 34 Diagnosis: severe mental illness as defined by the New York Office of Mental Health. Included: 62% schizophrenia, 24% depression/anxiety, 6% bipolar disorder Setting: a VR programme affiliated with an urban medical centre (Mount Sinai Hospital). All study participants were in a combined vocational and day treatment programme that focused on work and accepted only consumers with work goals. Available services included case management, pharmacological treatment, day treatment activities, housing support services, volunteer work at the site, paid internships at the hospital, and supported employment. Age: ≥ 18 years, mean age 44.0 years Gender: 59% male Ethnicity: 62% African American, 15% Hispanic Substance abuse: 26% current alcohol use disorder, 26% current drug use disorder Living situation:‐ Marital status: 76% never married Employment status: unemployed Working history: history of unsatisfactory job ending, defined as either being fired from a job or quitting a job prior to securing another job. Mean months since last job: 65 Motivation: interest in obtaining work Education: mean years of education: 12 years Disability benefit: ‐ Excluded:‐ Other: 74% 1 or more medical co‐morbidities. Most frequent: hypertension (29%), diabetes (19%), high cholesterol (19%), hepatitis C (15%)
Interventions	VR + cognitive training (N = 18) Participants were engaged in 24h of computer‐based cognitive exercises (Cogpack), which provided practice across the broad range of cognitive functions.Sessions required 45‐60 min, with consumers usually completing 2 sessions per week for about 16 weeks. Participants received performance scores to reinforce them. In addition, they participated in a weekly group. Topics in the group included the role of cognition in job performance and problem solving about compensatory strategies for dealing with common challenges on the job, such as remembering tasks, remaining focused, and improving work speed. Employment specialists were asked to attend the group if their consumer was having job performance difficulties. Plus VR: see below. VR only (N = 16) The vocational programme provided 2 types of services: internships and supported employment, each served by a separate team of vocational staff. The internship programme was an innovative VR model that provided work experience in time‐limited (up to 9 months), part‐time (up to 15 h), integrated (at the Mount Sinai Hospital) jobs, paying predominantly competitive wages or higher, depending on the participants’ ability to perform the job duties. Upon completing an internship work experience, consumers could choose another internship experience or SE SE was available to participants who had satisfactory performance in an internship job. The SE programme adhered to most of the principles, including integration of clinical and vocational services, matching jobs to consumers’ preferences, skills, and experiences, and ongoing, time‐unlimited support from employment specialists, who carried an average caseload of 25 consumers. The programme deviated from the zero exclusion and rapid job search principles of SE because consumers were encouraged to complete an internship job before enrolling in SE. In addition, job development and job support were provided by different vocational staff. No fidelity measurements described
Outcomes	Number of participants in competitive employment Weeks in competitive employment Mental health (PANNS)
Notes	SAMHSA definition of competitive employment used
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	"Participants were randomised by the project co‐ordinator using a computer‐generated randomisation programme"
Allocation concealment (selection bias)	Unclear risk	"Treatment assignment was not known in advance by study personnel"
Blinding of participants and personnel (performance bias)   All outcomes	Unclear risk	Participants and personnel could identify given intervention by content of programme
Blinding of outcome assessment (detection bias)   All outcomes	Unclear risk	Cognitive and Psychopathology assessments were conducted by an evaluator who was blind to treatment assignment. Employment activities were tracked weekly. No further details
Incomplete outcome data (attrition bias)   All outcomes	High risk	32 (94%) completed the 3‐month assessment, and 25 (74%) were followed up for 24 months. No further details. No reasons of missing data described
Selective reporting (reporting bias)	Low risk	All listed outcomes are reported
Other bias	Low risk	Funder: New York State Office of Mental Health

Open in a new tab

Michon 2014.

Methods	Allocation: randomised Design: multi centre Duration: 30 months Country: the Netherlands
Participants	N = 151 Diagnosis: severe mental illness. Included: 58% schizophrenia/psychotic disorder, 13% affective disorder, 22% personality disorder, 17% developmental disorder Setting: 4 regional CMH care divisions targeted at adult persons with severe mental illness Age: 18‐65 years, mean 34.9 years Gender: 74% male Ethnicity: 96% Dutch nationality Substance abuse: 11% drug (ab)use Living situation: 64% independent, 17% with family, 11% sheltered Marital status: 9% married Employment status: unemployed Working history: 61% paid employment in past 5 years, 53% competitive employment in past 5 years Motivation: explicit wish for competitive employment Education: 21% elementary school, 44% high school, 26% vocational education, 6% university degree Disability benefit: 58% disability benefit, 39% social benefit Excluded: paid work at study entrance, full‐time hospitalisation, engagement in another professional VR trajectory and participant in another study with conflicting interests
Interventions	IPS (N = 71) IPS was implemented according to protocol, with employment specialists added to multidisciplinary CMH teams with a staff:client ratio varying from 1:20 to 1:30. The majority of mental health services and treatment offered by these outpatient teams were provided in the community, employing assertive outreach. Team staff consisted of psychiatrists, psychologists, community psychiatric nurses and other personnel, for example, rehabilitation workers. In all services IPS workers assisted people in getting regular jobs, offered follow‐along support, spent most of the time in the community and operated in close collaboration with the other CMH team members. Model fidelity of IPS was evaluated at 3 time points using the Quality of Supported Employment Implementation Scale. Several research team members were trained in IPS fidelity assessment at Dartmouth Center. Each assessment was done by two researchers according to protocol; all scores were based on consensus. Two of the four participating agencies scored “good‐high” on fidelity in every assessment, and two scored “moderate.” TVR (N = 80) In general these services offer a stepwise vocational trajectory, putting much stronger emphasis on lengthy assessment of individual competencies and on connecting to prevocational activities such as voluntary jobs before placement in regular jobs. The TVR staff did not participate in the mental health teams.
Outcomes	Number of participants who obtained competitive employment Number of participants who obtained non‐competitive employment Days to first competitive employment Weeks in competitive employment Mental health (MHI‐5) Quality of life (MANSA) Hospital admissions Dropouts
Notes	Definition competitive employment: a paid job in a company or organisation in the regular labour market, against prevailing wages, not set aside for persons with a disability, that is, in an integrated work setting (one day or more)
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	"A stratified block randomisation procedure was used, with site and employment history (having had paid employment in the past 5 years yes or no) as stratification factors"
Allocation concealment (selection bias)	Low risk	"Randomisation was performed by an independent agency that sent the randomisation outcomes to the research team and the local research co‐ordinators at the same time"
Blinding of participants and personnel (performance bias)   All outcomes	High risk	Participants and personnel could identify given intervention by content of programme
Blinding of outcome assessment (detection bias)   All outcomes	Low risk	"Interviewers were trained by the research team and were blind to the conditions"
Incomplete outcome data (attrition bias)   All outcomes	High risk	Response at 30 months was 57%. IPS and TVR arms did not differ in response/non‐response ratio nor in non‐response reasons. Reasons described. ITT analyses carried out. In case of the primary outcome they gathered data even for people who dropped out, so there was only one missing value on the primary outcome variable (employment) regarding the 30‐month period
Selective reporting (reporting bias)	Low risk	All listed outcomes were reported
Other bias	Unclear risk	Supported by grants from UWV (national authority on employee insurances) and ZonMw (national funding of health research and development). Other sponsors were Trimbos Institute, UMCG‐RGOc (University Medical Center Groningen) and Internal funding UMCG. Each participant received EUR 10 per completed interview

Open in a new tab

Mueser 2004.

Methods	Allocation: randomised Design: single centre, multi‐arm Duration: 24 months Country: Hartford, Connecticut, USA
Participants	N = 204 (N = 135 included in this review) Diagnosis: severe mental illness, as defined by the State of Connecticut Department of Mental Health and Addiction Services (DSM IV Axis I diagnosis or borderline personality disorder and severe impairment in psychosocial functioning or self‐care) Included: 52% schizophrenia, 22% schizoaffective disorder, 6% bipolar disorder, 18% major depression Setting: CMH centre Age: mean 41.4 years Gender: 61% male Ethnicity: 44% African American, 31% Hispanic, 23% white Substance abuse: 12% alcohol use disorder, 19% drug use disorder Living situation: not reported Marital status: 70% never married Employment status: lack of competitive employment Working history: 36% competitive employment in past 5 years, 22% non‐competitive or sheltered work in past 5 years Motivation: desire for competitive work Education: 54% less than high school Disability benefit: not reported Excluded: not reported
Interventions	IPS (N = 68) Employment specialists serve on clients’ treatment teams alongside other members of the team. Each employment specialist provides the full range of vocational services to each client, including engagement in services, identifying job interests and vocational assessment, job finding, and job support. IPS uses assertive outreach, based on the ACT case‐management model for severe mental illness, to deliver most vocational services in clients’ natural settings in the community rather than at mental health or rehabilitation agencies. Fidelity to the IPS model was evaluated yearly by means of the IPS Fidelity Scale. Comparisons of the total fidelity score and the three sub scales indicated high fidelity (70/75). Psychosocial rehabilitation program (N = 67) The PSR program incorporated transitional employment into its vocational rehabilitation approach. Clients participated in a series of preparatory training activities, followed by transitional employment jobs and help them obtaining competitive work. In addition, the PSR program offered a drop‐in centre, skills training and support groups, recreational outings, and residential services. Overall, a survey suggested that the services and program philosophy of the PSR program in this study were typical of other PSR programs operating in the state of Connecticut during the time of the study. It should be noted that it was not certified by the International Center for Clubhouse Development. Standard services (N = 69)* Clients who were randomly assigned to standard services met with the vocational services coordinator and then were assigned to the program of their choice. This condition involved access to all other vocational services in Hartford for clients with severe mental illness, most of which were provided by one of two programs that contracted directly with the Connecticut Department of Mental Health and Addiction Services: a supported employment program located off‐site from the mental health centre (standard–supported), and a vocational program In which clients worked in jobs paying sub minimum wage or competitive wages in supervised janitorial enclaves in the community, fulfilling contracts obtained by that program (standard– enclave). Clients in standard services were also eligible to receive services from the Connecticut Bureau of Rehabilitation Services. * not included in this review because we could not classify this intervention
Outcomes	Percentage of participants who obtained competitive employment Percentage of participants who obtained non‐competitive employment Weeks in competitive employment Days to first competitive employment Mental health (PANNS) (no usable data) Dropouts
Notes	Defintion of competitive employment: paid competitive wages, job in an integrated setting in the community (i.e. afforded regular contact with nonclients), contracted by the client (i.e. not filled at the discretion of the vocational programme). Part of Employment Intervention Demonstration Project (EIDP)
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	"Clients were randomly assigned on the basis of a computer‐generated randomisation list, stratified by work history (competitive work in past 5 years or not), ethnicity, and gender"
Allocation concealment (selection bias)	Unclear risk	No details
Blinding of participants and personnel (performance bias)   All outcomes	High risk	Participants and personnel could identify the given intervention by contents of the programme
Blinding of outcome assessment (detection bias)   All outcomes	High risk	"Interviewers were not blind to vocational programme assignment"
Incomplete outcome data (attrition bias)   All outcomes	Low risk	Of the total sample 81% completed the 24‐month interview. Vocational data were missing for 6 participants; N = 3 IPS; N = 1 PSR and standard services N = 2. ITT analysis performed
Selective reporting (reporting bias)	Low risk	All listed outcomes reported
Other bias	Low risk	This project was one of eight controlled trials of VR programmes, supported by co‐operative #UD7 SM51818 from the US Department of Health and Human Services (DHHS) SAMHSA, Center for Mental Health Services (CMHS) as part of the Employment Intervention Demonstration programme (EIDP). Additional support was provided by National Institute of Mental Health (NIMH) Grants MH00842 and MH56147.

Open in a new tab

Nuechterlein 2012.

Methods	Allocation: randomised Design: single centre Duration: 18 months Country: Los Angeles, California, USA
Participants	N = 69 Diagnosis: recent onset of psychotic illness with the beginning of the first major psychotic episode occurring within last 2 years, a diagnosis of schizophrenia or schizoaffective disorder mainly schizophrenic subtype. Included: 83% schizophrenia, 17% schizoaffective disorder. Mean total time ill 24.6 months Setting: recruited from a variety of local psychiatric hospitals and psychiatric clinics and through referrals from the UCLA outpatient service. All participants were receiving outpatient psychiatric treatment at UCLA aftercare Research Program Age: 18‐45 years, mean 25.2 years Gender: 67% male Ethnicity: 29% white, rest Asian/Pacific, black or Hispanic Substance abuse: no drug abuse or alcoholism in 6 months prior to hospitalisation, no evidence that substance abuse will be a prominent factor in course of disease Living situation: ‐ Marital status: 93% single Employment status: ‐ Working history: ‐ Motivation: interest in resume work or school Education: mean 13.2 years Disability benefit: 1/4 were receiving disability funds Excluded: substance abuse, neurological disorder, premorbid mental retardation, no contraindication for risperidone
Interventions	IPS + Workplace Fundamentals Module (N = 46) The option of supported education was integrated with SE. IPS was combined with a group skills training approach, Workplace Fundamentals Module (WFM), to enhance return to regular school or competitive employment. The IPS worker facilitated a rapid search for schooling or employment, used assertive outreach, and gave ongoing vocational support. The same clinical team provided case management and psychiatric services for all participants. They adapted supported education to the IPS principles such that the whole programme met the standards for IPS fidelity. WFM is a group skills training approach that has the complementary goal of teaching social and problem solving skills necessary to keep a job. Each week for 6 months the participants were scheduled to attend two 75 min groups on the same day followed by 12 months of sessions on a fading frequency scale. The IPS worker would reinforce material from group WFM sessions by using it in the context of IPS meetings. Brokered VR (N = 23) Referrals to traditional VR services at separate agencies. The individual case manager discussed the range of options with each participant and agreed upon appropriate directions. They participated in skills training groups, but the groups did not focus on work settings and work skills. Their skills training included medication management training and communication skills training using methods that were similar to WFM
Outcomes	Percentage of participants who obtained competitive employment Dropouts
Notes	Outcome data derived from a systematic review (Bond 2015a) No IPS fidelity scores reported No further description of competitive employment
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	"Participants were randomly assigned in a 2/3 vs 1/3 ratio"
Allocation concealment (selection bias)	Unclear risk	No details
Blinding of participants and personnel (performance bias)   All outcomes	High risk	Participants and personnel could identify given intervention by contents of programme
Blinding of outcome assessment (detection bias)   All outcomes	High risk	No details
Incomplete outcome data (attrition bias)   All outcomes	High risk	74% follow‐up 10 lost to follow‐up in intervention group and 8 in control group
Selective reporting (reporting bias)	Unclear risk	No full report available
Other bias	Unclear risk	Funding source not described

Open in a new tab

O'Brien 2003.

Methods	Allocation: cluster‐randomised Design: 10 clusters, 6 intervention and 4 control Duration: 1 year Country: London, UK
Participants	N = 1037 Diagnosis: 54% psychosis, 21% depression/anxiety, 9% bipolar disorder Setting: 10 CMH teams of a large London Mental Health Trust Age: > 16 years, 72% ≥ 35 years Gender: 54% male Ethnicity: 63% white, 18% black Substance abuse:‐ Living situation: 90% living in the community Marital status: 57% single Employment status: unemployed Working history: 71% had been in open employment in the past Motivation:‐ Education:‐ Disability benefit:‐ Excluded: men > 65 years and women > 60 years (normal retirement age) were excluded
Interventions	Vocational staff training (N = 645) Three structured 1‐h seminars by a Consultant Clinical Psychologist specialising in employment issues and a Work Placement co‐ordinator. The seminars supplemented by a written handbook, addressed vocational assessment, access to work and education and vocational planning/intervention for longer term, unemployed CMH team participants. The second covered the best ways to match individuals' needs and wishes with the opportunities available. The third was about incorporating work and education targets as a routine in the care plans. Team members were also supplied with a directory of work and educational opportunities and services in the local area. Seminars were held at weekly intervals and were completed for all teams within 6 weeks of randomisation date. Standard care (N = 392) High quality standard CMH team care
Outcomes	Percentage of participants who obtained competitive employment Percentage of participants who obtained non‐competitive employment Hospital admissions Dropouts
Notes	Open/competitive employment, no definition but they distinguish sheltered work, voluntary work, education or training, work‐related activity and open employment
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Randomisation was at the level of the teams rather than the participant (cluster randomisation). Estimating an intra‐cluster co‐efficient of 0.02. Teams were randomised to control or intervention by an independent statistician. Recruitment before randomisation. They used 2 boroughs and teams were randomised within each borough to minimise geographical variations. Analyses without adjustment for cluster design, because the ICC was 0.00148, which leads to a design effect of 1.01.
Allocation concealment (selection bias)	Low risk	Randomisation was concealed
Blinding of participants and personnel (performance bias)   All outcomes	High risk	Personnel could identify given intervention by content of programme. Participants not described
Blinding of outcome assessment (detection bias)   All outcomes	High risk	The researchers could not be blinded to the intervention received because data had to be recorded from team case notes
Incomplete outcome data (attrition bias)   All outcomes	Low risk	Follow‐up data were obtained on 994 participants (95.8%). Records were inaccessible for 43 participants (27 intervention, 16 control). Of the 994 participants with follow‐up data, for 20 the keyworkers could not confirm vocational activity (15 intervention, 5 control). Vocational outcome data were obtained at 1 year on 974 participants (94% of the total sample). All analysis was carried out by the team and condition to which the subject was initially allocated. No loss of clusters reported
Selective reporting (reporting bias)	Low risk	All listed outcomes are reported
Other bias	Low risk	London Region Responsive Funding Programme At baseline the groups were comparable in terms of demographic and clinical characteristics. Only significant differences at baseline between individuals: ethnicity and patient status. These differences were controlled for in the final stratified analysis. Adjustment for baseline patient status and ethnicity showed that the initial differences did not effect outcome

Open in a new tab

Oshima 2014.

Methods	Allocation: randomised Design: single centre Duration: 6 months Country: Tokyo, Japan
Participants	N = 37 Diagnosis: primary diagnosis of either schizophrenia, mood disorder or neurotic disorder as defined by International Classification of Diseases (ICD)‐10. ≥ 5 contacts with the community support centre in the last year. High psychiatric service utilisation in last 2 years (≥ 2 or more hospitalisations, ≥ 100 inpatient days, ≥ 3 psychiatric emergency room visits, or ≥ 3 months no‐show to outpatient clinics). Low level of social functioning in the previous year as indicated by a score of ≤ 50 on the Global Assessment of Functioning Scale. Mean illness duration: 18 years Setting: a small community support centre administered by the city government in a suburban area near Tokyo. The centre staff provided broker‐style case management services. In close proximity to the centre were 2 other programmes: an ACT team funded through the national government and a Clubhouse operated by a private nonprofit organisation. These three organisations engaged in significant collaboration with each other Age: 18‐59 years, mean 40.6 years Gender: 49% male Ethnicity: ‐ Substance abuse: ‐ Living situation: 62% living with their family, 24% living alone Marital status: ‐ Employment status: not currently competitively employed Working history: 70% competitive employment before onset of illness, 35% after onset Motivation: ‐ Education:‐ Disability benefit:‐ Excluded: primary diagnosis of mental retardation, dementia, substance/alcohol abuse or personality disorder. No prior enrolment in supported employment
Interventions	IPS (N = 18) IPS employment specialists received 4 months of training from a team of IPS trainers. The employment specialists also received on‐the‐job supervision from 2 senior psychiatrists. The IPS programme adhered to IPS model standards. Employment specialists assessed each participant’s work preferences, past work experiences, current skills, and tolerance for type and intensity of job demands. Together, participants and specialists searched for competitive jobs. Specialists provided time‐unlimited support before, during, and after periods of employment. The overall fidelity rating was 68 out of 75, which is considered good implementation. TVR (N = 19) TVR services at the community support centre. The programme consisted of PVT in various work groups in a simulated environment intended to help prepare participants for paid employment
Outcomes	Percentage of participants who obtained competitive employment Percentage of participants who obtained non‐competitive employment Weeks in competitive employment Dropouts
Notes	The fidelity assessment was conducted by the first study author Competitive employment was operationally defined as a job paying at least minimum wage (as established in Japanese law), with ≥ five work h/week, for which anyone can apply, and not controlled by a service agency
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	Participants were randomised
Allocation concealment (selection bias)	Unclear risk	No details
Blinding of participants and personnel (performance bias)   All outcomes	High risk	Participants and personnel could identify given intervention by content of programme
Blinding of outcome assessment (detection bias)   All outcomes	High risk	"The research interviewers were not blinded to study condition." The employment outcomes were assessed through self‐report and cross‐checked through chart records, which were maintained by support centre staff, who had day‐to‐day contact with participant
Incomplete outcome data (attrition bias)   All outcomes	Low risk	No dropouts
Selective reporting (reporting bias)	Low risk	All listed outcomes reported
Other bias	Low risk	Supported by the Ministry of Health Labour and Welfare, Japan (Health and Labour Sciences Research Grant 200733005)

Open in a new tab

Penk 2010.

Methods	Allocation: randomised Design: single centre Duration: 12 months Country: Bedford, Massachussettes, USA
Participants	N = 89 Diagnosis: clinically stable veterans with co‐morbid psychiatric and substance use disorders: 56% major depression, 33% bipolar disorder, 25% panic disorder, 26% other anxiety disorder Setting: VA healthcare Services Age: < 56 years, mean 45.2 years Gender: 100% male Ethnicity: 78% white Substance abuse: current or lifetime drug or alcohol abuse or dependence with active substance abuse in the prior 90 days. Included: 88% alcohol abuse, 56% drug abuse Living situation: homeless Marital status: 26% never married Employment status: Working history: history of at least 1 day of competitive employment within the last 3 years Included: 56% full‐time employment in prior 3 years Motivation: a stated goal of returning to competitive employment within 6 months Education: mean 13 years Disability benefit: 29% Excluded: an average of no more than 4 h of clinical appointments per week, any legally mandated treatment that would make employment impossible. Did not meet criteria for schizophrenia, schizoaffective disorder, or unspecified psychosis
Interventions	Transitional work experience (N =50) The Bedford CWT programme offers TWE placements within companies in the community, as well as a few placements at the medical centre. Ninety percent of TWE placements exist in integrated, real‐work settings at which programme participants are doing the same work as non‐participants working alongside them. Job Placement services (N = 39) Participants assigned to job placement services were given contact numbers for 2 state VR specialists and were assisted in arranging the first appointment  The job placement providers were briefed as to the goal and design of the study and agreed to provide “typical” JP services for study participants. They discussed the employment goals and available resources. This service Included individual job search training and job support/coaching.
Outcomes	Percentage of participants who obtained competitive employment Percentage of participants who obtained non‐competitive employment Weeks in competitive employment Days to first competitive employment Dropouts
Notes	Participants who worked competitively at least 1 week
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	Participants were randomly assigned
Allocation concealment (selection bias)	Unclear risk	No details
Blinding of participants and personnel (performance bias)   All outcomes	High risk	Participants and personnel could identify programme assignment
Blinding of outcome assessment (detection bias)   All outcomes	Unclear risk	All data were collected by research assistants who were independent of the intervention staff. No details about blinding
Incomplete outcome data (attrition bias)   All outcomes	High risk	2 participants of the intervention group disappeared after the baseline evaluation. 10 of those assigned to the control condition subsequently entered the intervention services during the 12‐month follow‐up period. All stated that they entered the intervention services in order to obtain work because they felt that the control services were not helpful. Follow‐up data for participants assigned to intervention and who subsequently entered the intervention services were censored at the point of entry into intervention
Selective reporting (reporting bias)	Low risk	All listed outcomes reported
Other bias	Low risk	This material is based upon work supported by the Department of Veterans Affairs, Veterans Health Administration, Office of Research and Development, and the VA New England Mental Illness Research Education and Clinical Center

Open in a new tab

Schonebaum 2006.

Methods	Allocation: randomised Design: multi centre Duration: 30 months Country: Massachussetts, USA
Participants	N = 177 Diagnosis: bipolar disorder, major depression, or schizophrenia and its related disorders as defined by the DSM‐IV, included: 52% schizophrenia Setting: Genesis Club, an ICCD‐certified clubhouse and The PACT at Community Healthlink Age: ≥ 18 years, mean 38.1 years Gender: 55% male Ethnicity: 79% white Substance abuse: 35% substance abuse disorder Living situation: not reported Marital status: not reported Employment status: not competitively employed at time of intake Working history: 57% had a standard job in 5 years before study Motivation: they did not screen on work interest, but 70% interested in work Education: 63% high school diploma Disability benefit: not reported Excluded: severe mental retardation (IQ > 60) or previously participated in either programme
Interventions	Clubhouse (N = 89) A clubhouse run collaboratively by members and staff that emphasised mutual support, self‐determination, and therapeutic benefits of voluntary and paid work. Various rehabilitation services, including case management, a work‐ordered day, supported education, supported employment, transitional employment, and weekend social activities, were continuously available to clubhouse members, but attendance was not mandatory. The ICCD certified this programme and ensured fidelity to the standards for Clubhouse programmes PACT (N = 88) PACT was a mobile team that provided out‐of‐office clinical care, assistance with housing and daily living, substance abuse intervention, and help in finding meaningful activities or employment. Fidelity was verified annually. In both programmes, vocational staff who had training in SE worked closely with other staff to ensure rapid placement into mainstream jobs not reserved by employers for individuals with disabilities. On‐the‐job training and support were provided whenever needed. Clubhouse members could also work transitional employment jobs, which were above‐minimum‐wage jobs reserved for the clubhouse by a consortium of local employers. both programmes maintained acceptable fidelity over the 4‐year research period to SE model standards as assessed by a second SE expert
Outcomes	Percentage of participants who obtained competitive employment Number of weeks in competitive employment Dropouts
Notes	Part of Employment Intervention Demonstration Project (EIDP) Definition competitive employment: all jobs lasting at least 5 days that met the US Department of Labor’s definition of competitive employment: mainstream, integrated work paying at least minimum wage
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	"Participants were randomly assigned to one of the two conditions by picking a card from a hat"
Allocation concealment (selection bias)	Unclear risk	No details
Blinding of participants and personnel (performance bias)   All outcomes	High risk	Participants and personnel could identify given intervention by content of programme
Blinding of outcome assessment (detection bias)   All outcomes	High risk	"Staff at each programme tracked all jobs acquired by their participants, many of which were obtained with assistance from the respective agencies. Other employment data were self‐reported to programme staff or the interviewers"
Incomplete outcome data (attrition bias)   All outcomes	High risk	N = 167 of 177 (94%) participants were followed up after 24 months: 83 Clubhouse and 84 PACT. After 127 weeks, 72 Clubhouse and 76 PACT participants remained active in the project (148 of 177 participants = 84%) Reasons mentioned
Selective reporting (reporting bias)	Low risk	All listed outcomes were reported
Other bias	Unclear risk	This study was made possible by co‐operative grant UD7‐SM‐51831 from the Center for Mental Health Services as part of the Employment Intervention Demonstration Program. The views expressed in this article are solely the responsibility of the authors and do not necessarily represent the official views of any agencies or collaborators. Editor’s Note (Psychiatric Services): The papers by Macias and colleagues and by Schonebaum and coworkers were submitted independently a year apart and reviewed separately. Neither set of authors knew of the others’ efforts. The two articles used different methods and a different choice of variables but came to the same overall conclusion. Macias present the independent assessment of evaluators, whereas Schonebaum offer the perspective of researchers connected to Fountain House, which created the Clubhouse model. Each validates findings of the other, turning this coincidence into what we think is an interesting lesson.

Open in a new tab

Tsang 2001.

Methods	Allocation: cluster‐randomised Design: multi‐arm (3), multi centre (9) Duration: 3 months Country: Hong Kong, China
Participants	N = 97 Diagnosis: schizophrenia, no less than 1 year cumulative hospitalisation Setting: 9 community‐based, staffed residential facilities for ex‐mentally ill people. Participants were recruited from halfway houses and sheltered workshops Age: between 18‐50 years, mean 35.7 years Gender: 62% male Ethnicity: Hong Kong Chinese, who spoke Cantonese, not English, 98% of Hong Kong residents are Chinese Substance abuse: ‐ Living situation: staffed residential facilities Marital status: ‐ Employment status: unemployed, mean duration of unemployment 41 months Working history: eligibility criteria: previous occupation: blue collar, low‐level clerical, or service industry Motivation: willingness to participate in a work‐related social skills programme Education: no less than 5 years of primary school and no more than 5 years of secondary school, 30% finished primary school, 29% F1‐F3 (grade 7‐9), 35% F4‐F5 (grade 10‐11) Disability benefit: ‐. Excluded: learning disability
Interventions	Social skills training group (N = 26)* Hierarchical stages of learning were established based on a foundation of basic social skills and basic social survival skills followed by core work‐related skills, including those related to job securing and job retaining. The programme consisted of 10 weekly group sessions lasting 1.5‐2 h, with approximately 6‐8 members in each group. Each training group was facilitated by a trained occupational therapist assisted by an untrained welfare worker experienced in working with this client group. Social skills training group with follow‐up contacts (N = 30)* Received the social skills training plus follow‐up contact with group members and the trainer for 3 months gathered at a monthly meeting conducted by one of the occupational therapists who had run the training groups. These occasions were not as structured as the programme itself, and participants were encouraged to share their experiences of job hunting and job keeping. Control group (n = 41) Received standard psychiatric care on an outpatient basis
Outcomes	Percentage of participants who obtained competitive employment
Notes	*We merged the 2 social skills training groups together for the analyses 'Gainfully employed' seems to be competitive because of the job types reported: caretaker, security guard, waiter, junior clerk
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	"Randomisation was achieved by randomly assigning each of the 9 residential facilities to one of the three group conditions" Unclear if participants were recruited before or after randomisation. Design effect not reported
Allocation concealment (selection bias)	Unclear risk	No details
Blinding of participants and personnel (performance bias)   All outcomes	Low risk	"All participants were blind to research design. Participants did not know that there were groups with and without follow‐up."
Blinding of outcome assessment (detection bias)   All outcomes	Low risk	"The raters were blind to study design and the group status of the participants"
Incomplete outcome data (attrition bias)   All outcomes	Unclear risk	No details, but big numbers are unlikely because of short follow‐up
Selective reporting (reporting bias)	Low risk	All listed outcomes were reported
Other bias	Unclear risk	No details about study funding

Open in a new tab

Tsang 2010.

Methods	Allocation: randomised Design: multi‐arm, multicentre Duration: 39 months for both intervention groups, 15 months for the control group Country: Hong Kong, China
Participants	N = 189 Diagnosis: severe mental illness (operationally defined as schizophrenia, schizoaffective disorder, bipolar disorder, recurrent major depression, or borderline personality disorder); at least 2 years of major role dysfunction; medium–high functioning and free from serious role dysfunction for the past 3 months Included: 77% schizophrenia Setting: 2 non‐government organisations and 3 day hospitals in Hong Kong Age: mean 34.9 years Gender: 49% male Ethnicity: ‐ Substance abuse: ‐ Living situation: ‐ Marital status: ‐ Employment status: unemployed Working history: 91% employment history Motivation: desire to work Education: 77% secondary education Disability benefit: ‐ Excluded: memory impairment, learning disorder, and neurological or medical illness that would preclude their working or participation in assessments
Interventions	IPS (N = 65) A single‐minded focus on competitive employment; eligibility for services based solely on client choice, with no exclusion on the basis of work readiness, substance use problems, lack of motivation, treatment non‐compliance and so on; rapid job search upon programme admission using the ‘place then train’ approach; attention to client preferences in the job search, rather than dependence on job availability; close integration between the employment services and the mental health treatment team; ongoing, individualised support and job training after clients obtained employment; systematic benefits counselling; and consultation with employer or job supervisor including advocacy accommodations. IPS fidelity scores ranged from 66‐68 out of 75 (88%–91%) Integrated SE (N = 58) The integrated SE programme combined IPS and WSST. The main difference with IPS is that it was enhanced by 10‐session WSST. The social skill training was provided to integrated SE participants before obtaining employment. A problem‐solving approach was used to help participants handle interpersonal conflicts throughout the follow‐up period. The IPS fidelity scores ranged from 65‐68 out of 75 (87%–91%). TVR (N = 66) TVR participants received comprehensive vocational assessments and pre‐vocational training conducted in the VR centres. Vocational assessments included work samples, vocational interest exploration, and situational vocational assessments. After the establishment of participants’ baseline work performance, pre‐vocational training on entry‐level job tasks were provided in order to help them develop specific job skills and work habits. The participants were placed in a sheltered environment in various work groups such as clerical training, computer training, and cleaning training. The aim of the workshop‐based training was to promote the participants to sheltered workshop or competitive employment.
Outcomes	Percentage of participants who obtained competitive employment Weeks in competitive employment Quality of life (PWI) Dropouts
Notes	Success in competitive employment was defined as having continuously worked in the job for at least 2 months for at least 20 h/week Data for the TVR group were available up to 15 months. They did not collect further follow‐up data due to the limited improvement in vocational and/or non‐vocational outcomes.
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	The participants were randomly assigned using SPSS
Allocation concealment (selection bias)	Unclear risk	No details
Blinding of participants and personnel (performance bias)   All outcomes	High risk	"The employment specialists were not blind to the treatment assignment of the participants." Participants could identify the given intervention by contents of the programme
Blinding of outcome assessment (detection bias)   All outcomes	Low risk	"Assessments were conducted by an independent, trained, and blind assessor who was a registered occupational therapist"
Incomplete outcome data (attrition bias)   All outcomes	High risk	N = 85 (69%) completed the follow‐up assessment at 39 months N = 44/58 integrated SE participants and N =41/65 IPS participants. 54 TVR participants (81.8%) completed the 15‐month follow‐up assessment. ITT analyses were conducted on the entire randomised sample (N = 189)
Selective reporting (reporting bias)	Low risk	All listed outcomes are reported
Other bias	Low risk	This study was funded by a grant from the Health Services Research Committee (HHSRF Project No. 03040031) of the Hong Kong Government

Open in a new tab

Twamley 2012a.

Methods	Allocation: randomised Design: single centre Duration: 12 months Country: San Diego, USA
Participants	N = 58 Diagnosis: 40% schizophrenia and 60% schizoaffective disorder Setting: CMH clinic Age: ≥ 45 years, mean 51.0 years Gender: 64% male Ethnicity: 60% white Substance abuse: 22% substance/alcohol use during study Living situation: ‐ Marital status: ‐ Employment status: unemployed Working history: 79% without significant paid work in 2 years, but 86% had once worked at least 12 months continuously, mean years since last job: 7 Motivation: stating a goal of working Education: mean 12.4 years Disability benefit: mean monthly disability entitlement income USD 830 Excluded: substance abuse/dependence within 30 days, history of head injury with loss of consciousness > 30 min, mental retardation, or neurological disorders
Interventions	IPS (N= 30) Participants received manualised SE from an employment specialist whose maximum caseload was 25. IPS emphasises competitive work, integrated mental health and SE services, any client can participate, rapid job searching, service‐users' preferences, time‐unlimited follow‐along support, benefits counselling, and providing services in community settings. IPS fidelity ratings improved from “fair” to “good” over the study. “High” fidelity could not be achieved due to study design (only schizophrenia/schizoaffective clients included; study duration was 1 year; only 1 employment specialist) TVR (N = 28) Participants were referred to the Department of Rehabilitation for orientation, intake, and eligibility determination, then became clients of a brokered programme for individuals with mental illness. Vocational counsellors carried caseloads of 35 clients; additional staff provided job‐readiness and prevocational coaching/classes. To promote engagement and reduce attrition, study staff assisted participants with appointment‐setting, reminder calls, and transportation, if needed, to the first 3 appointments
Outcomes	Percentage of participants who obtained competitive employment Percentage of participants who obtained non‐competitive employment Weeks in competitive employment Days to first competitive employment Dropouts
Notes	Competitive work was defined as employment paying at least minimum wage and not reserved for the disabled
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	Randomised. No further details
Allocation concealment (selection bias)	Unclear risk	No details
Blinding of participants and personnel (performance bias)   All outcomes	High risk	Participants and personnel could identify the given intervention by contents of programme
Blinding of outcome assessment (detection bias)   All outcomes	High risk	Raters were blinded. Employment outcomes (primary outcome) were collected via self‐report
Incomplete outcome data (attrition bias)   All outcomes	High risk	N = 46 were followed up (79%). 12 participants (IPS = 7; CVR = 5) dropped out, no significant difference between groups All analyses were ITT. Dropouts were assumed not to work, zeros were imputed for employment data following dropout
Selective reporting (reporting bias)	Low risk	All listed outcomes reported
Other bias	Low risk	This work was supported by the National Institute of Mental Health (MH066011 to EWT) and the National Alliance for Research on Schizophrenia and Depression; neither the NIMH nor NARSAD had any further role in study design; in the collection, analysis and interpretation of data; in the writing of the report; or in the decision to submit the paper for publication

Open in a new tab

Viering 2015.

Methods	Allocation: randomised Design: single centre Duration: 24 months Country: Zürich, Switzerland
Participants	N = 250 (183 unemployed at baseline) Diagnosis: mental disorder, all participants had to be in psychiatric and/or psycho‐therapeutical treatment during the whole study period. Included: 47% mood disorder, 16% schizophrenia/schizoaffective disorder, 17% personality disorder. Severity: > 50% 1‐5 admissions and mean age onset disease 10 years earlier than age at baseline Setting: University Hospital of Psychiatry Zurich (PUK) Age: 18‐60 years, mean 42.6 years Gender: 47% male Ethnicity: 78% born in country of residence Substance abuse: ‐ Living situation: lived in the canton of Zurich, 53% living alone, 36% with friends/relatives Marital status: ‐ Employment status: 73% unemployed Working history: 97% work history: 50% worked 1 month in past years, 47% < 1 month Motivation: wish to enter the competitive employment market or to remain there if they already had a job Education: 67% basic school, 15% high school, mean 10.2 years Disability benefit: IV‐pension due to a mental disorder (full or part time pension), < 1 year Excluded: organic mental disorder, mental retardation
Interventions	IPS (N = 127, N = 92 unemployed at baseline) The intervention relied on the SE approach IPS. In total, there were 4 job coaches enrolled. 2 of them were full‐time employed, the other part time. All of them had a degree in psychology. The coaching frequency and the coaching duration of each session were determined individually by the job coach and the individual. No training of abilities or social skills nor any assessments of skills were administered beforehand. The job coach gave support during the application procedure (e.g. establishing realistic goals, writing applications, preparation of the job interview), and continued providing support according to the IPS principles during the participant employment (e.g. how to cope with workplace stressors including interpersonal conflicts with colleagues). The support was continued also in cases of job loss. The IPS fidelity scale was administered every 3 months. Moderate IPS fidelity was given throughout the whole study period (M = 61.2). Two items, item 4 (“cooperation with other institutions and other care team individuals”) and 14 (“community‐oriented services”), were rated low. Prevocational rehabilitation (N = 123, N = 91 unemployed at baseline) Participants of the control group were free to choose other vocational services including PVT, but were not supported by a job coach from ZhEPP
Outcomes	Percentage of participants who obtained competitive employment Dropouts
Notes	The primary outcome, obtaining competitive employment was scored as successfully fulfilled if the job was obtained by standard procedures (written application, CV, and job interview) and if the job was kept for at least 1 month We used the data of the participants who were unemployed at baseline (N = 183)
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	"For the purpose of randomisation, a list of numbers was created based on a Bernoulli distribution, a form of binomial probability distribution. Each participant was randomised according to that list"
Allocation concealment (selection bias)	Unclear risk	No details
Blinding of participants and personnel (performance bias)   All outcomes	High risk	Participants and personnel could identify given intervention by content of programme
Blinding of outcome assessment (detection bias)   All outcomes	High risk	No details about blinding. The participants were interviewed every 6 months by research workers
Incomplete outcome data (attrition bias)   All outcomes	High risk	The overall dropout rate was 32% (N = 79/250). The dropout rates in both groups were similar. 17 participants dropped out, from the participants who had a job in the beginning. In each group 31 participants who were unemployed at baseline dropped out: 62/183 = 34% Data for all participants were analysed as ITT. For that, the last observation carried forward (LOCF) method was used, meaning that in case of dropout, the last observation of a participant was used to replace the missing value.
Selective reporting (reporting bias)	Low risk	Study protocol was available, not all secondary outcomes were reported, it’s is possible that a separate publication will show those results
Other bias	Low risk	This study was funded by the Swiss Social Insurance Office (BSV). This funding was used for the salary of the job coaches and the scientific personnel and for the compensation of the interviews. No money was used to amplify participants’ income. The funding source had no influence on the design and the implementation of the study.

Open in a new tab

Waghorn 2014.

Methods	Allocation: randomised Design: multi centre Duration: 12 months Country: Brisbane, Townsville and Cairns, Australia
Participants	N = 208 Diagnosis: psychotic disorder and a consumer of the mental health service at time of referral, not in an acute phase of illness and considered by the clinical team to be able to safely participate in the programme. Included: 81% psychotic disorder, 8% bipolar disorder, 6% major depressive disorder or anxiety disorder Setting: 4 CMH services Age: 18‐59 years, mean 32.4 years Gender: 69% male Ethnicity: Australian residents, 88% English spoken at home Substance abuse: ‐ Living situation: living in the mental health service catchment area with no immediate plans to move Marital status: 89% not partnered/single Employment status: not currently employed and not employed within the previous 3 months, currently available to work for ≥ 8 h/week Working history: 32% employed in prior year Motivation: interest in competitive employment as a goal Education: 68% completed year 12 Disability benefit: 68% disability support pension, 13% unemployment benefit Excluded: ‐
Interventions	IPS (N = 106) The intervention was governed by a standardised service level agreement between two agencies that enabled a full‐time employment specialist employed by the employment service, to be co‐located into the mental health team as the sole person delivering the employment service to volunteer consenting consumers of the mental health service. All four employment service partners in the intervention condition were contracted to the Australian Government and all received recurrent case‐based funding. The employment specialist was typically co‐located at the mental health service 4 of 5 d/week. Overall, the 4 interventions achieved good fidelity to IPS principles in the range 66–73 Disability employment services (N = 102) Non‐integrated forms of SE known as Disability Employment Services in Australia. Each site (intervention and control condition) received an initial training in evidence‐based practices in SE. Mental health case managers were given responsibility for delivering the control condition as part of enhanced routine mental health case management. They were provided assistance to engage with disability employment services in the local area. They were provided with a resource guide supported to select suitable employment services. Regular communication with the employment specialist was then encouraged. At 2 sites (Townsville and Cairns), there were no other disability employment services willing to accept referrals from the mental health teams, so staff of the same employment service but not co‐located with the mental health team, provided the control service. Employment services accepting these referrals had the same service contracts with the Australian Government. The four controls were estimated at fair IPS fidelity (range 56–65)
Outcomes	Percentage of participants who obtained competitive employment Weeks in competitive employment Dropouts
Notes	Competitive employment was defined as jobs in the open labour market at award wages or above, and not temporary jobs, piece work, voluntary work or unpaid work experience, and not jobs reserved for people with disabilities. The trial involved 4 other sites that did not use a randomised controlled design. One site did not succeed in implementation. Neither a full‐time co‐located employment specialist nor a local steering group, were established. Consequently, only fair fidelity was achieved with respect to IPS practices. Data collection was discontinued after 6 months because the arrangements resembled an informal collaboration more than a formal partnership. Data from this site were excluded from this analysis in the article.
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Sample randomisation was conducted centrally for all sites, at an individual level to a 1:1 allocation ratio, using a purpose‐designed MSAccess randomisation algorithm
Allocation concealment (selection bias)	Low risk	Central allocation
Blinding of participants and personnel (performance bias)   All outcomes	High risk	Due to the visible nature of the employment services being provided, it was not possible to mask the results of randomisation
Blinding of outcome assessment (detection bias)   All outcomes	Unclear risk	No details
Incomplete outcome data (attrition bias)   All outcomes	High risk	Early drop outs 28.4%. The majority of these continued in the vocational service but refused to take part in the data collection interviews. Attrition cases were more likely to speak a different language at home but otherwise did not differ from those that remained in the study. 56% completed follow‐up interview, N = 67 of intervention group and N = 49 in control group, reasons mentioned in flow chart
Selective reporting (reporting bias)	Low risk	All listed outcomes were reported
Other bias	High risk	Implementation was based on a formal partnership approach that links a CMH service to an Australian Government‐funded disability employment service. Funding was offered in the form of a non‐recurrent AUD 75,000, per site, in the first year and AUD 50,000 in the second year. It was agreed that any participants who objected to their control condition randomisation would be offered the opportunity of transferring to the intervention after a minimum of 6 months’ assignment to the control service. Subsequently, 28 participants originally allocated to the control service opted to transfer to the IPS intervention after 6 months. The contaminated results were retained in the control group as originally allocated, but these results were also analysed separately.

Open in a new tab

Walker 1969.

Methods	Allocation: randomised Design: single centre Duration: 6 months Country: Brockton, Massachusetts, USA
Participants	N = 28 Diagnosis: clearance by staff physician for off‐grounds rehabilitation. Included: 50% schizophrenia, 18% anxiety disorder, 14% depression Setting: VA hospital Age: ‐ Gender: 96% male Ethnicity:‐ Substance abuse: 7% alcohol, no further details Living situation: inpatient Marital status:‐ Employment status: unemployed Working history: minimal 2 weeks prior to study successful community hospital industrial rehabilitation placement, recommendation by rehabilitation therapist Motivation: willingness to volunteer for community CHIRP Education:‐ Disability benefit:‐ Excluded:‐
Interventions	Community‐based hospital industrial rehabilitation placement (CHIRP) (N = 14) Placements in a regular industrial setting off grounds (form of paid sheltered workshop), supervision by member of rehabilitation staff from hospital, transport, could continue to attend after leaving hospital, and standard hospital and community care. Standard care (N = 14) Standard hospital and community care
Outcomes	Number of participants who obtained competitive employment Number of participants who obtained non‐competitive employment Hospital admissions
Notes	Definition competitive employment: obtaining "regular competitive employment". Patients were free to secure regular employment or to continue in CHIRP during the study period
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	"Use of a table of random numbers"
Allocation concealment (selection bias)	Unclear risk	Participants were taken into the study in pairs (intervention and control)
Blinding of participants and personnel (performance bias)   All outcomes	High risk	Participants and personnel could identify given intervention by content of programme
Blinding of outcome assessment (detection bias)   All outcomes	High risk	No details about blinding
Incomplete outcome data (attrition bias)   All outcomes	Low risk	Data were obtained for all participants
Selective reporting (reporting bias)	Low risk	All outcomes were reported
Other bias	Unclear risk	No details about funding source. Study authors were working at the hospital

Open in a new tab

Wong 2008.

Methods	Allocation: randomised Design: single centre Duration: 18 months Country: Hong Kong, China
Participants	N = 92 Diagnosis: diagnosed at least 2 years previously as having a mental illness. Included: 70% schizophrenia spectrum, 18% affective disorder Setting: the Occupational Therapy Department, Kwai Chung Hospital Age: between 18‐55 years, mean 33.6 years Gender: 60% male Ethnicity: ‐ Substance abuse: ‐ Living situation: ‐ Marital status: 90% never married Employment status: unemployed Working history: mean months working at a paid job in the past 5 years: 17 months Motivation: expressed an interest in competitive employment Education: 68% grade 7‐11, 22% post secondary (grade 12 or above) Disability benefit: ‐ Excluded: serious medical illness that might affect their long‐term ability in competitive work
Interventions	SE (N = 46) The SE programme of this study was based on the IPS model. The employment specialist was integrated into the participant’s clinical management team. The employment specialist assisted the participant to search for a competitive job on the basis of his or her educational background, work preference, and previous work experience. Once employed, on‐the‐job training and follow‐along support was provided. If employment was terminated for any reason, the employment specialist would assist the participant to recover from job loss and help him or her look for another job. The maximum caseload was limited to 20. The IPS programme developers were invited to conduct a 1‐week intensive training on the programme implementation as well as to rate the present programme by using the IPS Fidelity Scale. The local IPS programme received high ratings of implementation fidelity (scored 69 out of 75, which is equivalent to good implementation of SE). TVR (N = 46) The programme was implemented in the form of PVT in various work groups in a simulated environment. The primary objective of this programme was to equip participants with skills and knowledge related to choosing, obtaining, and keeping a competitive job in the community by using a stepwise train‐place approach. Although the participants were attending the programme, they were encouraged to seek open competitive employment by themselves by using the normal channels for job hunting, such as newspaper advertisements and Internet searches, as well as personal contacts with potential employers.
Outcomes	Percentage of participants who obtained competitive employment Percentage of participants who obtained non‐competitive employment Weeks in competitive employment Days to first competitive employment Dropouts
Notes	Definition competitive employment: a job paid at the market rate, for which anyone can apply, and not controlled by a service agency
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	"Participants were individually randomly assigned by using random numbers generated by computer"
Allocation concealment (selection bias)	Unclear risk	No details
Blinding of participants and personnel (performance bias)   All outcomes	High risk	Participants and personnel could identify the given intervention by contents of the programme
Blinding of outcome assessment (detection bias)   All outcomes	High risk	Measurements were conducted by the employment specialists
Incomplete outcome data (attrition bias)   All outcomes	Low risk	1 participant in the control group was lost to follow‐up
Selective reporting (reporting bias)	Low risk	All listed outcomes were reported
Other bias	Low risk	This study was supported by grant 216033 from the Health Care and Promotion Fund, Food and Health Bureau, Hong Kong

Open in a new tab

Xiang 2007.

Methods	Allocation: randomised Design: single centre Duration: 2 years Country: Bejing, China
Participants	N = 103 Diagnosis: inpatients with a diagnosis of schizophrenia according to DSM–IV. They had been clinically stable for at least 1 month before recruitment and were about to begin their pre‐discharge home leave. Mean duration of illness 15 years Setting: the Chaoyang Mental Health Care Institute. As a district psychiatric hospital it has inpatient and outpatient service patients with schizophrenia. Age: 18‐60 years, mean 38.6 years Gender: 47% male Ethnicity: ‐ Substance abuse: excluded Living situation: at least one family member would be cohabiting with the participant after discharge Marital status: 54% never married Employment status: no employment immediately after discharge Working history: ‐ Motivation: ‐ Education: mean 10.7 years Disability benefit: ‐ Excluded: the presence of ongoing acute medical or neurological conditions, and current or a history of misuse of drugs and substances other than nicotine
Interventions	Community re‐entry module (N = 53) A module of a standardised, structured social skills training programme devised at the University of California, Los Angeles. The community re‐entry module was primarily designed for inpatients, to foster seamless care in the transition between hospital and community. It consisted of 16 training sessions. Each of the sessions was taught using the 7 learning activities described in the trainer’s manual: introduction; videotape and questions/answers; role‐play; resource management; outcome problems; in vivo exercises; homework assignments. Each group comprised 6‐8 participants and the group sessions took place 4 times/week. Group psycho‐education (N = 50) An equally intensive programme of group psycho‐education, a standard psychosocial intervention in many parts of China. The opportunity to attend quarterly, community‐based workshops following discharge was offered to participants in both study groups as part of a routine intervention to reinforce the use in the community of skills acquired during admission. In addition, family members were encouraged to participate in these regular workshops.
Outcomes	Percentage of participants who obtained competitive employment Mental health (PANNS) Hospital admissions Dropouts
Notes	Re‐employment was defined as at least 3 consecutive months of salaried employment during study period
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	Participants were randomly allocated
Allocation concealment (selection bias)	Unclear risk	No further details
Blinding of participants and personnel (performance bias)   All outcomes	High risk	Participants and personnel could identify intervention allocation by components of programme
Blinding of outcome assessment (detection bias)   All outcomes	Low risk	Two raters independently assessed all participants. The raters were not involved in the interventions and were also masked to the study protocol. Before commencing the study, participants were instructed by the research co‐ordinator not to disclose their group membership to the raters at any stage of study. In order to assess the effectiveness of the raters’ masking, we designed a 5‐point Likert scale. These results showed that the raters were not sure about the participants’ group membership, suggesting that masking was maintained relatively successfully throughout the study period
Incomplete outcome data (attrition bias)   All outcomes	Low risk	94/103 (91%) participants completed all the research interviews. Missing post‐intervention and follow‐up data were calculated by using the ‘replace missing calculated by using the ‘replace missing values’ option of the SPSS
Selective reporting (reporting bias)	Low risk	All listed outcomes were reported
Other bias	Low risk	The study was funded by two grants, ZD199816 and TS199801, from the Bureau of Public Health, Beijing

Open in a new tab

ACT: assertive community treatment; BPRS: Brief Psychiatric Rating Scale; CBT: cognitive behavioural therapy; CMH: community mental health; CWT: compensated work therapy; DPA: diversified placement approach DSM: Diagnostic and Statistical Manual of Mental Disorders; GED: General Education Diploma; HADS: Hamilton Depression and Anxiety Scale; IPS: individual placement and support; ITT: intention‐to‐treat; ISA: integrated service agency; IV: invalidity; MANSA: Manchester Short Assessment of Quality of Life; MHI: Mental Health Inventory; MMSE: Mini‐Mental State Examination; PACT: programme of ACT; PANNS: PTSD: post‐traumatic stress disorder; PVT: prevocational training; PWI: Personal Wellbeing Index; QOLI: The Quality of Life Interview; QOLP: Lancashire Quality of Life Profile; SAMHSA: Substance Abuse and Mental Health Services Administration; SE: supported employment; SFHS: Short‐Form Health Survey SMD: serious mental disorder; SMI: severe mental illness; SPSS: Statistical Package for the Social Sciences; SSA: Social Security Administration; SSDI; Social Security Disability Insurance; SSI; Supplemental Security Income; TVR: traditional vocational rehabilitation; TWE: transitional work experience; VA: Veterans Affairs; VR: vocational rehabilitation; WSST: work‐related social skills training

Characteristics of excluded studies [ordered by study ID]

Study	Reason for exclusion
Audini 1994	Unclear employment outcome
Bateman 1999	Wrong population: majority employed at baseline
Bayer 2008	Large group of participants was already employment at baseline (43%)
Becker 2007	Wrong design: participants from 2 studies, not a RCT
Bell 1996	No competitive employment outcomes
Bell 2003	Paid work was part of the intervention
Bell 2005	No competitive employment outcomes
Bell 2008b	Transitional funds were used
Bell 2014	Transitional funds were used, analysis of two trials
Bertelsen 2008	No competitive employment outcomes
Bond 2016	Wrong design: secondary analysis on a pooled sample from four RCTs
Cook 2005	This is an aggregation of 8 independent RCTs from different cities in the USA. This project was named the Employment Intervention Demonstration Program (EIDP). We used the individual reports of these RCTs because of the heterogeneity of the interventions and control conditions.
Cook 2009	No competitive employment outcomes
Davis 2012	Wrong population: veterans with PTSD. Participants with schizophrenia, schizo‐affective disorder and bipolar disorder excluded because they were already eligible for IPS
Davis 2015	All participants were placed in jobs at baseline
Fowler 2009	No competitive employment outcomes
Granholm 2014	Employment was a combination of volunteer and paid employment
Griffiths 1974	Unclear definition of employment
Hamilton 2000	No competitive employment outcomes
Hasslet 2014	No competitive employment outcomes
Hirschfeld 2002	No competitive employment outcomes
Hogarty 2004	No competitive employment outcomes
Jager 2013	Follow‐up of Zurich sample after termination of the EQOLISE trial (Burns 2007)
Kidd 2014	No competitive employment outcomes
Kline 1981	Unclear definition of employment
Kopelowicz 1998	No competitive employment outcomes
Kuldau 1977	No competitive employment outcomes
Kurtz 2013	No competitive employment outcomes
Kurtz 2015	No competitive employment outcomes
Liberman 1998	No competitive employment outcomes
Lindenmayer 2008	No competitive employment outcomes
Lucca 2004	Wrong design: not a RCT
Luo 1994	Wrong design: not a RCT
Lysaker 2005	No competitive employment outcomes
Lysaker 2009	No competitive employment outcomes
Man 2012	Unclear definition of employment
McFarlane 2015	Quasi experimental design, assignment based on clinical risk (severity of positive symptoms)
McGrew 2005	Wrong design: a mixed randomised and quasi experimental design was used
McGurk 2003	Wrong design: not randomised controlled
Mueser 2005	All participants were employed at baseline
Mueser 2011	Secondary analysis of results of 4 RCTs
Okpaku 1997	No competitive employment outcomes
Resnick 2008	The definition of employment did not differentiate between competitive or transitional/sheltered employment
Rinaldi 2010	Wrong design: cohort study
Roder 2002	Wrong design: matching procedure
Rogers 2006	Unclear number of participants employed at baseline
Rosen 2014	wrong population: majority employed or in military service at baseline
Rus 2013	No competitive employment outcomes
Sato 2014	Not randomised, the assignment was based on the order of entry
Shi 2002	No competitive employment outcomes
Sungur 2011	No competitive employment outcomes
Swildens 2011	No competitive employment outcomes
Thunissen 2008	Wrong population: majority was employed at baseline
Torrent 2013	No competitive employment outcomes
Trapp 2013	Wrong design: not RCT. A big part of the participants (42%) was employed at baseline. No competitive employment outcomes
Tsang 2013	No competitive employment outcomes
Twamley 2005	Wrong design: includes data of a retrospective study
Twamley 2012b	No competitive employment outcomes
Vauth 2005	No competitive employment outcomes
Wolkon 1971	No competitive employment outcomes
Xiang 2006	Wrong population: majority employed at baseline

Open in a new tab

PTSD: post‐traumatic stress disorder; RCT: randomised controlled trial

Characteristics of studies awaiting assessment [ordered by study ID]

Bejerholm 2017.

Methods	RCT
Participants	Adults, aged 18‐63 years, with a depressive episode, recurrent depression or bipolar disorder and interested in employment and not employed during the past year
Interventions	Individual Enabling and Support (IES) compared to TVR IES is guided by an employment specialist who works closely with the participant in relation to the outpatient team, family, Social Insurance Agency, Public Employment Service and employers. There are ten IES principles: 1) handling change and developing motivational and cognitive strategies, 2) having a time‐use pattern that supports work‐life balance. Principles 3–10 correspond to the IPS model. TVR was delivered by various professionals in several settings and organisations and was regulated by the social benefit and unemployment security system. The first step involved reducing symptoms and increasing work ability at a mental health service. Step 2 involved assessment of 50% work capacity and was performed by the SIA and PES. If work capacity was not met, participants could enter Step 3 with pre‐vocational activities at the municipality. The last step was vocational training during internship placements, and these could lead to employment positions through the Public Employment Service.
Outcomes	Number of participants, weeks and hours a week in competitive employment, internship, education, PVT. Depressive severity and quality of life
Notes

Open in a new tab

Glynn 2017.

Methods	2 x 2 RCT
Participants	People with serious mental illness
Interventions	Work skills training programme and IPS compared to IPS, and the participants were also randomly assigned to risperidon or olanzapine
Outcomes	Clinical and work outcomes
Notes

Open in a new tab

Kane 2015.

Methods	Cluster‐RCT
Participants	People, 15‐40 years, with first episode psychosis and ≤ 6 months on antipsychotic medication
Interventions	NAVIGATE compared to usual community treatment. NAVIGATE included 4 interventions: personalised medication management, family psycho education, resilience‐focused individual therapy and SE plus education. Usual community treatment was psychosis treatment determined by clinician choice and service availability
Outcomes	Days of paid employment or school attendance, employment rate, employment earnings, disability income, public support. Other outcomes published elsewhere: quality of life, mental health
Notes

Open in a new tab

McGurk 2015.

Methods	RCT
Participants	People with severe mental illness (schizophrenia, bipolar disorder, major depression) who had not obtained or kept competitive work despite receiving high‐fidelity SE
Interventions	All participants continued to receive their usual mental health services, including a SE programme enhanced by training employment specialists in the management of cognitive impairments. Participants were randomly assigned to receive enhanced SE only or enhanced SE plus the Thinking Skills for Work program, which used three approaches: cognitive exercise practice (COGPACK), strategy coaching, and teaching coping/compensatory strategies
Outcomes	Cognitive outcomes, competitive work outcomes, symptoms and quality of life
Notes

Open in a new tab

McGurk 2016.

Methods	RCT
Participants	Participants with SMI (83% schizophrenia) who had not benefited from vocational rehabilitation
Interventions	Vocational services enhanced by training vocational specialists in recognising cognitive difficulties and providing job‐relevant cognitive coping strategies (Enhanced Vocational Rehabilitation), or similarly enhanced vocational services and cognitive remediation (Thinking Skills for Work). Participants were recruited form 3 employment programmes: Project Moving On, MetroClub, and Adult Rehabilitation Services. Each of these programmes had a unique vocational rehabilitation model.
Outcomes	Employment outcomes including competitive employment, clinical symptoms
Notes

Open in a new tab

Schneider 2016.

Methods	RCT
Participants	People aged 18–60 years, and on the caseload of the rehabilitation and recovery or early intervention in psychosis teams
Interventions	All participants received IPS. In the intervention group participants were offered 3‐6 sessions of work‐focused counselling delivered by a psychologist based on generic psychological practice, including goal‐based motivational procedures and CBT.
Outcomes	Hours in paid employment and other vocational activities such as education, training or volunteer work. Self‐esteem, costs, health and well‐being, self‐assessed barriers to work, perceived stigma, avoidance of social disapproval, social cognition and social problem solving
Notes

Open in a new tab

CBT: cognitive behavioural therapy  IPS: individual placement and support  PVT: prevocational training  RCT: randomised controlled trial  SE: supported employment  SMI: serious mental illness  TVR: traditional vocational rehabilitation

Characteristics of ongoing studies [ordered by study ID]

Bell 2015.

Trial name or title	Cognitive training to enhance work program outcomes: preliminary findings
Methods	RCT
Participants	People with psychotic disorders participating in all types of work programmes including incentive therapy, CWT and SE
Interventions	Cognitive training PositScience or Nintento BrainAge. BrainAge employed engaging cognitive game software and PositScience used specially designed exercises that narrowly focus on discrete cognitive processes. Work services were provided as usual
Outcomes	Neurocognitive, vocational and quality of life outcomes
Starting date	Unknown
Contact information	Unknown
Notes	No data available yet

Open in a new tab

Bitter 2015.

Trial name or title	Effectiveness of the Comprehensive Approach to Rehabilitation (CARe) methodology: design of a cluster randomized controlled trial
Methods	Cluster‐RCT
Participants	People with severe mental illness from participating organisation for sheltered and supported housing facilities
Interventions	Teams in the intervention group receive the CARe methodology training. The aims of this training are: training workers in the principles of rehabilitative and recovery‐supportive care and to support clients' rehabilitation process in a methodical way. The study will give special attention to the process of implementation. The CARe methodology consists of 6 steps: (1) building and maintaining a constructive relationship; (2) collecting information and making a personal profile with the client; (3) helping the client with formulating wishes, making choices and setting goals; (4) helping the client to making a Personal Plan; (5) helping the client execute the plan and (6) following the process, learn, evaluate and adjust. After the training programme the workers will be supported in working according to the CARe methodology by means of CARe coaching meetings (once every 4‐6 weeks). The teams in the control group do not receive the CARe methodology training. The workers in those teams will maintain work according to the (narrowly implemented) outdated CARe methodology.
Outcomes	Primary outcomes: recovery, social functioning (including employment status) and quality of life. Other outcomes are empowerment, hope, self‐efficacy beliefs and need for care. Model fidelity audit will be performed
Starting date	May 2012
Contact information	n.a.bitter@uvt.nl
Notes	Unclear if competitive employment outcomes will be presented

Open in a new tab

Christensen 2015.

Trial name or title	Individual placement and support supplemented with cognitive remediation and work‐related social skills training in Denmark: study protocol for a randomized controlled trial
Methods	Randomised, three‐arm, assessor‐blinded, multi centre trial
Participants	Adults diagnosed with schizophrenia, schizotypal or delusional disorders, bipolar disorder, or severe depression. Participants must reside in one of two major Danish cities: Copenhagen or Odense. They must be assigned to early intervention teams or community mental health services. They must express a clear desire for competitive employment or education.
Interventions	IPS and services as usual or IPS enhanced with cognitive remediation and work‐related social skills training and services as usual
Outcomes	The primary outcome is “hours in competitive employment or education”. Danish employment legislation provides opportunities for financial support when obtaining competitive employment. This could be subsidised employment. Secondary outcomes are work or education at some point during the follow‐up period (yes/no), days to first employment or beginning of education, cognitive impairment, functional level, self‐esteem, and self‐efficacy
Starting date	October 2012
Contact information	thomas.03.christensen@regionh.dk
Notes

Open in a new tab

Granholm 2015.

Trial name or title	Enhancing assertive community treatment with cognitive behavioral social skills training for schizophrenia: study protocol for a randomized controlled trial
Methods	RCT
Participants	People diagnosed with schizophrenia or schizoaffective disorder, receiving ACT services for at least 3 months, no prior social skills training or CBT in the past 3 years, and living in the community for at least the past month
Interventions	ACT alone or ACT + Adapted Cognitive Behavioral Social Skills Training (CBSST) The CBSST intervention is delivered in the context of the regular ACT visits. CBSST integrates CBT and social skills training techniques.
Outcomes	The primary outcome domain is psychosocial functioning: everyday living skills and activities related to employment, education, and housing. Additional outcome domains of interest include mediators of change in functioning, symptoms, quality of service and programme fidelity.
Starting date	February 2012
Contact information	egranholm@ucsd.edu
Notes	This is also a mixed method implementation study

Open in a new tab

Harris 2015.

Trial name or title	Internet based cognitive remediation can assist people with severe mental illness to gain and retain employment‐the CogRem study
Methods	RCT
Participants	Unemployed people with severe mental illness who had joined a supported employment programme
Interventions	Internet‐based, cognitive remediation (CogRem) compared to internet‐based information control (WebInfo)
Outcomes	Paid working hours, earnings
Starting date	Unknown
Contact information	Unknown
Notes	Information based on conference abstract

Open in a new tab

Melau 2011.

Trial name or title	The effect of five years versus two years of specialised assertive intervention for first episode psychosis ‐ OPUS II: study protocol for a randomized controlled trial
Methods	RCT
Participants	People, aged 18‐37 years, with first episode psychosis in the schizophrenia spectrum
Interventions	A 2‐year specialised intensive assertive treatment programme (OPUS) or standard treatment. The integrated OPUS treatment consists of 3 core elements; ACT, family treatment and social skills training
Outcomes	Negative symptoms, simultaneous remission of psychotic and negative symptoms, substance abuse, user satisfaction, adherence to treatment, compliance with medication, suicidal behaviour, working alliance, self‐efficacy, use of bed days, ability to live independently, and labour‐market affiliation
Starting date	July 2009
Contact information	marianne.melau@regionh.dk
Notes	We excluded the previous study (OPUS I trial) because they did not report competitive employment outcomes

Open in a new tab

Nordt 2012.

Trial name or title	'Placement budgets' for supported employment‐‐improving competitive employment for people with mental illness: study protocol of a multicentre randomized controlled trial
Methods	Multicentre RCT
Participants	Current treatment in one of the 6 participating outpatient psychiatric clinics 12 months of unemployment and no programme of vocational integration over the last 3 months Motivation to work in competitive employment Being of working age (18–60 years) Resident in the Canton of Zurich Willing and capable of giving informed consent
Interventions	IPS with three different placement budgets of 25 h, 40 h, or 55 h working hours of job coaches. Support lasts 2 years for those who find a job. The intervention ends for those who fail to find competitive employment when the respective placement budgets run out.
Outcomes	Time between study inclusion and first competitive employment that lasted ≥ 3 months, motivation, stigmatisation, social network and social support, quality of life, job satisfaction, financial situation, and health conditions
Starting date	June 2010
Contact information	cnordt@bli.uzh.ch
Notes

Open in a new tab

Sveinsdottir 2014.

Trial name or title	Protocol for the effect evaluation of Individual Placement and Support (IPS): a randomised controlled multicenter trial of IPS versus treatment as usual for patients with moderate to severe mental illness in Norway
Methods	Multicentre RCT
Participants	People currently undergoing treatment for moderate to severe mental illness, who are currently out of the labour market but have an expressed desire to work
Interventions	IPS or high quality treatment as usual. Treatment as usual involves being offered a prioritised spot in a vocational rehabilitation scheme, primarily Work with assistance and/or traineeship in a sheltered business
Outcomes	The primary outcome of the study is increased labour market participation in ordinary paid employment or education. The secondary outcomes are mental health status, disability and quality of life
Starting date	Unknown
Contact information	Vigdis.Sveinsdottir@uni.no
Notes	There will be a subgroup analysis for severe mental illness vs moderate mental illness

Open in a new tab

ACT: assertive community treatment  CBT: cognitive behavioural therapy  CWT: compensated work therapy  IPS: individual placement and support  RCT: randomised controlled trial  SE: supported employment

Differences between protocol and review

We improved the search strategy compared to the version described in the protocol (Suijkerbuijk 2015). We will use this new strategy also for the first update of this review in the future.

We used the outcomes hospital admissions and drop outs as adverse events instead of drop outs alone.

We used Tau² to assess heterogeneity for the network meta‐analysis.

We used the approach described by Salanti 2014 for grading the quality of evidence.

Instead of calculating the NNTB we calculated the corresponding risks of the interventions.

Contributions of authors

Conceiving the protocol/review: YS, FS

Designing the protocol/review: YS, FS, AO

Co‐ordinating the protocol/review: FS

Designing search strategies: YS, FS

Writing the protocol/review: YS, FS

Providing general advice on the protocol/review: JA, MC, JM

Sources of support

Internal sources

No sources of support supplied

External sources

Instituut Gak, Netherlands.

Amsterdam Public Health Research Institute was subsidised to conduct this Cochrane review as part of a larger research project regarding work and severe mental illness.

Declarations of interest

Yvonne Suijkerbuijk: None known.

Joost van Mechelen: None known.

Anneli Ojajärvi: None known.

Johannes Anema: I am a consultant for Evalua Nederland BV, that sells a health checklist to large employers and insurance companies to improve work ability and reduce future work disability. I also own Evalua Nederland BV stocks. I regularly receive travel and accommodation costs and sometimes speaker fees to present research results, including those of this review. I supervise a PhD student funded by a grant from Instituut GAK. I have also received a grant and travel support from Instituut GAK. I am an editor of the Handbook of Work Disability that includes part of the review results. I hold a chair (0.4 FTE) endowed by the Dutch Workers Compensation board.

Marc Corbiere: None known.

Frederieke G Schaafsma: None known.

New

References

References to studies included in this review

Au 2015 {published data only}

Au DW, Tsang HW, So WW, Bell MD, Cheung V, Yiu MG, et al. Effects of integrated supported employment plus cognitive remediation training for people with schizophrenia and schizoaffective disorders. Schizophrenia Research 2015;166(1):297‐303. [DOI] [PubMed] [Google Scholar]

Beard 1963 {published data only}

Beard JH, Malamud TJ, Rossman E. Psychiatric rehabilitation and long‐term rehospitalisation rates: the findings of two research studies. Schizophrenia Bulletin 1978;4:622‐35. [DOI] [PubMed] [Google Scholar]
Beard JH, Pitt MA, Fisher SH, Goertzel V. Evaluating the effectiveness of a psychiatric rehabilitation program. American Journal of Orthopsychiatry 1963;33:701‐12. [DOI] [PubMed] [Google Scholar]

Becker 1967 {published data only}

Becker RE. An evaluation of a rehabilitation program for chronically hospitalised psychiatric patients. Social Psychiatry 1967;2:32‐8. [Google Scholar]

Bejerholm 2015 {published data only}

Areberg C, Bejerholm U. The effect of IPS on participants' engagement, quality of life, empowerment, and motivation: a randomized controlled trial. Scandinavian Journal of Occupational Therapy 2013;20(6):420‐8. [DOI] [PubMed] [Google Scholar]
Bejerholm U, Areberg C, Hofgren C, Sandlund M, Rinaldi M. Individual placement and support in Sweden ‐ a randomized controlled trial. Nordic Journal of Psychiatry 2015;69(1):57‐66. [DOI] [PubMed] [Google Scholar]

Blankertz 1996 {published data only}

Blankertz L, Robinson S. Adding a vocational focus to mental health rehabilitation. Psychiatric Services 1996;47:1216‐22. [DOI] [PubMed] [Google Scholar]

Bond 1986 {published data only}

Bond GR, Dincin J. Accelerating entry into transitional employment in a psychosocial rehabilitation agency. Rehabilitation Psychology 1986;31:143‐55. [Google Scholar]

Bond 1995 {published data only}

Bond GR, Dietzen LL, McGrew JH, Miller LD. Accelerating entry into supported employment for persons with severe psychiatric disabilities. Rehabilitation Psychiatry 1995;40:75‐94. [Google Scholar]
Bond GR, Dietzen LL, Vogler K, Katuin CH, McGrew JH, Miller LD. Toward a framework for evaluating cost and benefits of psychiatric rehabilitation: three case examples. Journal of Vocational Rehabilitation 1995;5:75‐88. [Google Scholar]

Bond 2007 {published data only}

Bond GR, Salyers MP, Dincin J, Drake R, Becker DR, Fraser VV, et al. A randomised controlled trial comparing two vocational models for persons with severe mental Illness. Journal of Consulting and Clinical Psychology 2007;75(6):968‐82. [DOI] [PubMed] [Google Scholar]
Fraser VV, Jones AM, Frounfelker R, Harding B, Hardin T, Bond GR. VR closure rates for two vocational models. Psychiatric Rehabilitation Journal 2008;31(4):332‐9. [DOI] [PubMed] [Google Scholar]
Harding B, Torres‐Harding S, Bond GR, Salyers MP, Rollins AL, Hardin T. Factors associated with early attrition from psychosocial rehabilitation programs. Community Mental Health Journal 2008;44(4):283‐8. [DOI] [PubMed] [Google Scholar]
Kukla M, Bond GR. A randomised controlled trial of evidence based supported employment: nonvocational outcomes. Journal of Vocational Rehabilitation 2013;38:91‐8. [Google Scholar]
Kukla M, Bond GR. The working alliance and employment outcomes for people with severe mental illness enrolled in vocational programs. Rehabilitation Psychology 2009;54(2):157‐63. [DOI] [PubMed] [Google Scholar]

Bond 2015b {published data only}

Bond GR, Kim SJ, Becker DR, Swanson SJ, Drake RE, Krzos IM, et al. A controlled trial of supported employment for people with severe mental illness and justice involvement. Psychiatric Services 2015;66(10):1027‐34. [DOI] [PubMed] [Google Scholar]

Burns 2007 {published data only}

Burns T, Catty J. IPS in Europe: the EQOLISE trial. Psychiatric rehabilitation journal 2008;31(4):313‐17. [DOI] [PubMed] [Google Scholar]
Burns T, Catty J, Becker T, Drake, RE, Fioritti A, Knapp M, et al. The effectiveness of supported employment for people with severe mental illness: a randomised controlled trial in six European countries. The Lancet 2007;370:1146–52. [DOI] [PubMed] [Google Scholar]
Burns T, Catty J, White SJ, Becker T, Koletsi M, Fioritti A, et al for the EQOLISE Group. The impact of supported employment and working on clinical and social functioning: results of an international study of individual placement and support. Schizophrenia Bulletin 2009;35(5):949–58. [DOI] [PMC free article] [PubMed] [Google Scholar]
Burns T, White SJ, Catty J, for the EQOLISE Group. Individual placement and support in Europe: the EQOLISE trial. International Review of Psychiatry 2008;20(6):498‐502. [DOI] [PubMed] [Google Scholar]
Catty J, Lissouba P, White SJ, Becker T, Drake RE, Fioritti A, et al on behalf of the EQOLISE Group. Predictors of employment for people with severe mental illness: results of an international six‐centre randomised controlled trial. The British Journal of Psychiatry 2008;192:224‐31. [DOI] [PubMed] [Google Scholar]
Catty J, White SJ, Koletsi M, Becker T, Fioritti A, Kalkan R, et al for the EQOLISE group. Therapeutic relationships in vocational rehabilitation: predicting good relationships for people with psychosis. Psychiatry Research 2011;187(1):68‐73. [DOI] [PubMed] [Google Scholar]
Kilian R, Lauber C, Kalkan R, Dorn W, Rössler W, Wiersma D, et al. The relationships between employment, clinical status, and psychiatric hospitalisation in patients with schizophrenia receiving either IPS or a conventional vocational rehabilitation programme. Social Psychiatry and Psychiatric Epidemiology 2012;47(9):1381‐9. [DOI] [PubMed] [Google Scholar]
Knapp M, Patel A, Curran C, Latimer E, Catty J, Becker T, et al. Supported employment: cost‐effectiveness across six European sites. World Psychiatry 2013;12(1):60‐8. [DOI] [PMC free article] [PubMed] [Google Scholar]
Koletsi M, Niersman A, Busschbach Van JT, Catty J, Becker T, et al for the EQOLISE Group. Working with mental health problems: clients’ experiences of IPS, vocational rehabilitation and employment. Social Psychiatry and Psychiatric Epidemiology 2009;44(11):961‐70. [DOI] [PubMed] [Google Scholar]

Burns 2015 {published data only}

Burgess J, Yeeles K, Burns T. Refining Individual Placement and Support (IPS) and establishing its effectiveness: a pragmatic, non‐inferiority RCT (IPS‐LITE trial). Psychiatrische Praxis 2011;38:congress abstract. [Google Scholar]
Burns T, Yeeles K, Langford O, Montes MV, Burgess J, Anderson C. A randomised controlled trial of time‐limited individual placement and support: IPS‐LITE trial. The British Journal of Psychiatry 2015;207:351‐6. [DOI] [PubMed] [Google Scholar]

Chandler 1996 {published data only}

Chandler D, Hu TW, Meisel J, McGowen M, Madison K. Mental health costs, other public costs, and family burden among mental health clients in capitated integrated service agencies. Journal of Mental Health Administration 1997;24:178‐88. [DOI] [PubMed] [Google Scholar]
Chandler D, Meisel J, Hu TW, McGowen M, Madison K. A capitated model for a cross section of severely mentally ill clients: employment outcomes. Community Mental Health Journal 1997;33:501‐16. [DOI] [PubMed] [Google Scholar]
Chandler D, Meisel J, Hu TW, McGowen M, Madison K. Client outcomes in a three‐year controlled study of an integrated service agency model. Psychiatric Services 1996;47:1337‐43. [DOI] [PubMed] [Google Scholar]
Chandler D, Meisel J, Hu TW, Mcgowen M, Madison K. A capitated model for a cross‐section of severely mentally ill clients: hospitalization. Community Mental Health Journal 1998;34(1):13‐26. [DOI] [PubMed] [Google Scholar]
Chandler D, Meisel J, McGowen M, Mintz J, Madison K. Client outcomes in two model capitated integrated service agencies. Psychiatric Services 1996;47:175‐80. [DOI] [PubMed] [Google Scholar]

Craig 2014 {published data only}

Craig T, Shepherd G, Rinaldi M, Smith J, Carr S, Preston F, et al. Vocational rehabilitation in early psychosis: cluster randomised trial. British Journal of Psychiatry 2014;205(2):145‐50. [DOI] [PubMed] [Google Scholar]

Dincin 1982 {published data only}

Bond GR. An economic analysis of psychosocial rehabilitation. Hospital and Community Psychiatry 1984;35:356‐62. [DOI] [PubMed] [Google Scholar]
Dincin J, Witheridge TF. Psychiatric rehabilitation as a deterrent to recidivism. Hospital and Community Psychiatry 1982;33:645‐50. [DOI] [PubMed] [Google Scholar]

Drake 1996 {published data only}

Becker DR, Drake RE, Bond GR, Xie H, Dain BJ, Harrison K. Job terminations among persons with severe mental illness participating in supported employment. Community Mental Health Journal 1998;34(1):71‐82. [DOI] [PubMed] [Google Scholar]
Clark RE. Supported employment and managed care: can they coexist?. Psychiatric Rehabilitation Journal 1998;22(1):62‐68. [Google Scholar]
Drake RE, McHugo GJ, Becker DR, Anthony WA, Clark RE. The New Hampshire study of supported employment for people with severe mental illness. Journal of Consulting and Clinical Psychology 1996;64(2):391‐9. [DOI] [PubMed] [Google Scholar]
Mueser KT, Becker DR, Torrey WC, Xie H, Bond GR, Drake REJet al. Work and non‐vocational domains of functioning in persons with severe mental illness: a longitudinal analysis. Journal of Nervous and Mental Disease 1997;185(7):419‐26. [DOI] [PubMed] [Google Scholar]
Torrey WC, Mueser KT, McHugo GH, Drake RE. Self‐esteem as an outcome measure in studies of vocational rehabilitation for adults with severe mental illness. Psychiatric Services 2000;51(2):229‐33. [DOI] [PubMed] [Google Scholar]

Drake 1999b {published data only}

Dixon L, Hoch JS, Clark R, Bebout R, Drake R, McHugo G, et al. Cost‐effectiveness of two vocational rehabilitation programs for persons with severe mental illness. Psychiatric Services 2002;53(9):1118‐24. [DOI] [PubMed] [Google Scholar]
Drake RE, McHugo GJ, Bebout RR, Becker DR, Harris, M, Bond GR, et al. A randomised clinical trial of supported employment for inner‐city patients with severe mental disorders. Archives of General Psychiatry 1999;56(7):627–33. [DOI] [PubMed] [Google Scholar]

Drake 2013 {published data only}

Drake RE, Frey W, Bond GR, Goldman HH, Salkever D, Miller A, et al. Assisting Social Security Disability Insurance beneficiaries with schizophrenia, bipolar disorder, or major depression in returning to work. American Journal of Psychiatry 2013;170(12):1433‐41. [DOI] [PubMed] [Google Scholar]
Frey WD, Azrin ST, Goldman HH, Kalasunas S, Salkever DS, Miller AL, et al. The mental health treatment study. Psychiatric Rehabilitation Journal 2008;31(4):306‐12. [DOI] [PubMed] [Google Scholar]
Salkever DS, Gibbons B, Drake RE, Frey WD, Hale TW, Karakus M. Increasing earnings of Social Security Disability Income beneficiaries with serious mental disorder. The Journal of Mental Health Policy and Economics 2014;17(2):75‐90. [PubMed] [Google Scholar]

Drebing 2005 {published data only}

Drebing CE, Ormer EA, Krebs C, Rosenheck R, Rounsaville B, Herz L, et al. The impact of enhanced incentives on vocational rehabilitation outcomes for dually diagnosed veterans. Journal of Applied Behavioral Analysis 2005;38(3):359‐72. [DOI] [PMC free article] [PubMed] [Google Scholar]

Drebing 2007 {published data only}

Drebing CE, Ormer EA, Mueller L, Hebert M, Penk WE, Petry NM, et al. Adding contingency management intervention to vocational rehabilitation: outcomes for dually diagnosed veterans. Journal of Rehabilitation Research & Development 2007;44(6):851‐66. [DOI] [PubMed] [Google Scholar]

Eack 2009 {published data only}

Eack SM, Greenwald DP, Hogarty SS, Cooley SJ, DiBarry AL, Montrose DM, et al. Cognitive enhancement therapy for early‐course schizophrenia: effects of a two‐year randomized controlled trial. Psychiatric Services 2009;60(11):1468‐76. [DOI] [PMC free article] [PubMed] [Google Scholar]
Eack SM, Greenwald DP, Hogarty SS, Keshavan MS. One‐year durability of the effects of cognitive enhancement therapy on functional outcome in early schizophrenia. Schizophrenia Research 2010;120(1):210‐16. [DOI] [PMC free article] [PubMed] [Google Scholar]
Eack SM, Hogarty GE, Cho RY, Prasad KM, Greenwald DP, Hogarty SS, et al. Neuroprotective effects of cognitive enhancement therapy against gray matter loss in early schizophrenia: results from a 2‐year randomized controlled trial. Archives of General Psychiatry 2010;67(7):674‐82. [DOI] [PMC free article] [PubMed] [Google Scholar]
Eack SM, Hogarty GE, Greenwald DP, Hogarty SS, Keshavan MS. Cognitive enhancement therapy improves emotional intelligence in early course schizophrenia: preliminary effects. Schizophrenia Research 2007;89(1):308‐11. [DOI] [PMC free article] [PubMed] [Google Scholar]
Eack SM, Hogarty GE, Greenwald DP, Hogarty SS, Keshavan MS. Effects of cognitive enhancement therapy on employment outcomes in early schizophrenia: results from a 2‐year randomized trial. Research on Social Work Practice 2011;21(1):32‐42. [DOI] [PMC free article] [PubMed] [Google Scholar]
Eack SM, Pogue‐Geile MF, Greenwald DP, Hogarty SS, Keshavan MS. Mechanisms of functional improvement in a 2‐year trial of cognitive enhancement therapy for early schizophrenia. Psychological Medicine 2011;41(6):1253‐61. [DOI] [PMC free article] [PubMed] [Google Scholar]
Keshavan MS, Eack SM, Wojtalik JA, Prasad KM, Francis AN, Bhojraj TS, et al. A broad cortical reserve accelerates response to cognitive enhancement therapy in early course schizophrenia. Schizophrenia Research 2011;130(1):123‐9. [DOI] [PMC free article] [PubMed] [Google Scholar]
Lewandowski KE, Eack SM, Hogarty SS, Greenwald DP, Keshavan MS. Is cognitive enhancement therapy equally effective for patients with schizophrenia and schizoaffective disorder?. Schizophrenia Research 2011;152(2):291‐4. [DOI] [PubMed] [Google Scholar]

Gervey 1994 {published data only}

Gervey R, Bedell JR. Supported employment in vocational rehabilitation. Psychological Assessment and Treatment of Persons with Severe Mental Disorders. Washington DC: Taylor & Francis, 1994:170‐5. [Google Scholar]

Gold 2006 {published data only}

Gold PB, Meisler N, Santos AB, Carnemolla MA, Williams OH, Keleher J. Randomized trial of supported employment integrated with assertive community treatment for rural adults with severe mental illness. Schizophrenia Bulletin 2006;32(2):378‐95. [DOI] [PMC free article] [PubMed] [Google Scholar]

Hoffmann 2012 {published data only}

Hoffmann H, Jäckel D, Glauser S, Kupper Z. A randomised controlled trial of the efficacy of supported employment. Acta Psychiatrica Scandinavica 2012;125(2):157‐67. [DOI] [PubMed] [Google Scholar]
Hoffmann H, Jäckel D, Glauser S, Mueser KT, Kupper Z. Long‐term effectiveness of supported employment: 5‐year follow‐up of a randomized controlled trial. American Journal of Psychiatry 2014;171(11):1183‐90. [DOI] [PubMed] [Google Scholar]

Howard 2010 {published data only}

Heslin M, Howard L, Leese M, McCrone P, Rice C, Jarrett M, et al. Randomized controlled trial of supported employment in England: 2 year follow‐up of the Supported Work And Needs (SWAN) study. World Psychiatry 2011;10(2):132‐7. [DOI] [PMC free article] [PubMed] [Google Scholar]
Howard LM, Heslin M, Leese M, McCrone P, Rice C, Jarrett M, et al. Supported employment: randomised controlled trial. British Journal of Psychiatry 2010;196(5):404‐11. [DOI] [PMC free article] [PubMed] [Google Scholar]

Killackey 2008 {published data only}

Baksheev GN, Allott K, Jackson HJ, McGorry PD, Killackey E. Predictors of vocational recovery among young people with first‐episode psychosis: findings from a randomized controlled trial. Psychiatric Rehabilitation Journal 2012;35(6):421‐7. [DOI] [PubMed] [Google Scholar]
Killackey E, Jackson HJ, McGorry PD. Vocational intervention in first‐episode psychosis: individual placement and support v. treatment as usual. British Journal of Psychiatry 2008;193(2):114‐20. [DOI] [PubMed] [Google Scholar]

Killackey 2014 {published data only}

Allott KA, Cotton SM, Chinnery GL, Baksheev GN, Massey J, Sun P, et al. The relative contribution of neurocognition and social cognition to 6‐month vocational outcomes following individual placement and support in first‐episode psychosis. Schizophrenia Research 2013;150(1):136‐43. [DOI] [PubMed] [Google Scholar]
Killackey E, Allot K, Cotton SM, Jackson H, Scutella R, Tseng YP, et al. A randomized controlled trial of vocational intervention for young people with first‐episode psychosis: method. Early Intervention in Psychiatry 2013;7:329‐37. [DOI] [PubMed] [Google Scholar]
Killackey E, Allot KA, Cotton SM, Chinnery GL, Sun P, Collins Z, et al. Recovery in first‐episode psychosis: first results from a large randomized controlled trial of IPS. Early Intervention in Psychiatry. 2012; Vol. 6, issue Suppl. 1:13.
Killackey E, Allott KA, Cotton S, Chinnery GL, Jackson H. Baseline to 18 months: main results from a randomized controlled trial of individual placement and support for young people with first‐episode psychosis. Early Intervention in Psychiatry. 2014; Vol. 8, issue Suppl. 1:152.

Latimer 2006 {published data only}

Latimer EA, LecomteT, Becker DR, Drake RE, Duclos I, Piat M, et al. Generalisability of the individual placement and support model of supported employment: results of a Canadian randomised controlled trial. British Journal of Psychiatry 2006;189:65‐73. [DOI] [PubMed] [Google Scholar]

Lecomte 2014 {published data only}

Lecomte T, Corbière M, Lysaker PH. A group cognitive behavioral intervention for people registered in supported employment programs: CBT‐SE [Une intervention cognitive comportementale de groupe pour les personnes suivies dans le cadre d’un programme de soutien en emploi (TCC‐SE)]. L’Encéphale 2014;40:S81‐S90. [DOI] [PubMed] [Google Scholar]

Lehman 2002 {published data only}

Gold JM, Goldberg RW, McNary SW, Dixon LB, Lehman AF. Cognitive correlates of job tenure among patients with severe mental illness. American Journal of Psychiatry 2002;159:1359‐402. [DOI] [PubMed] [Google Scholar]
Lehman AF, Goldberg R, Dixon LB, McNary S, Postrado L, Hackman A, et al. Improving employment outcomes for persons with severe mental illnesses. Archives of General Psychiatry 2002;59(2):165‐72. [DOI] [PubMed] [Google Scholar]

McFarlane 1996 {published data only}

McFarlane WR, Dushay RA, Stastny P, Deakins SM, Link B. A comparison of two levels of family‐aided assertive community treatment. Psychiatric Services 1996;47:744‐50. [DOI] [PubMed] [Google Scholar]

McFarlane 2000 {published data only}

McFarlane WR, Dushay RA, Deakins SM, Stasny P, Lukens EP, Toran J, et al. Employment outcomes in family‐aided assertive community treatment. Journal of Orthopsychiatry 2000;70(2):203‐14. [DOI] [PubMed] [Google Scholar]

McGurk 2007 {published data only}

McGurk SR, Mueser KT, Pascaris A. Cognitive training and supported employment for persons with severe mental illness: one‐year results from a randomized controlled trial. Schizophrenia Bulletin 2005;31(4):898‐909. [DOI] [PubMed] [Google Scholar]
McGurk, SR, Mueser KT, Feldman K, Wolfe R, Pascaris A. Cognitive training for supported employment: 2‐3 year outcomes of a randomized controlled trial. American Journal of Psychiatry 2007;164:437‐41. [DOI] [PubMed] [Google Scholar]

McGurk 2009 {published data only}

McGurk SR, Mueser KT, DeRosa TJ, Wolfe R. Work, recovery, and comorbidity in schizophrenia: a randomized controlled trial of cognitive remediation. Schizophrenia Bulletin 2009;35(2):319‐35. [DOI] [PMC free article] [PubMed] [Google Scholar]

Michon 2014 {published data only}

Michon H, Busschbach JT, Stant AD, Vugt MD, Weeghel J, Kroon H. Effectiveness of individual placement and support for people with severe mental illness in the Netherlands: a 30‐month randomized controlled trial. Psychiatric Rehabilitation Journal 2014;37(2):129‐36. [DOI] [PubMed] [Google Scholar]
Stant AD, Busschbach JT, Vugt M, Michon H. A rehabilitation intervention to help people with severe mental illness obtain and keep a paid job: the economic evaluation. Journal of Mental Health Policy and Economics 2013;6:S32. [Google Scholar]
Busschbach JT, Michon H, Vugt M, Stant AD, Aerts‐Roorda LC, Erp N. Effectiveness of individual placement and support in the Netherlands. Final report of a randomised controlled trial. Part 2: Findings after 30 months follow‐up [Effectiviteit van Individuele Plaatsing en Steun in Nederland. Eindverslag van een gerandomiseerde gecontroleerde effectstudie Deel 2: Bevindingen na 30 maanden follow‐up]. Rob Giel Onderzoekcentrum, Groningen & Trimbos‐instituut, Utrecht (Dutch research report) 2011.

Mueser 2004 {published data only}

Mueser KT, Becker DR, Wolfe R. Supported employment, job preferences, job tenure and satisfaction. Journal of Mental Health 2001;10(4):411‐17. [Google Scholar]
Mueser KT, Bond GR, Essock SM, Clark RE, Carpenter‐Song E, Drake RE, et al. The effects of supported employment in Latino consumers with severe mental illness. Psychiatric Rehabilitation Journal 2014;37(2):113‐22. [DOI] [PubMed] [Google Scholar]
Mueser KT, Clark RE, Haines M, Drake RE, McHugo GJ, Bond GR, et al. The Hartford study of supported employment for persons with severe mental illness. Journal of Consulting and Clinical Psychology 2004;72(3):479‐90. [DOI] [PubMed] [Google Scholar]
Mueser KT, Essock SM, Haines M, Wolfe R, Xie H. Posttraumatic stress disorder, supported employment, and outcomes in people with severe mental illness. CNS Spectrums 2004;9(12):913‐25. [DOI] [PubMed] [Google Scholar]

Nuechterlein 2012 {published data only}

Bond GR, Drake RE, Luciano A. Employment and educational outcomes in early intervention programmes for early psychosis: a systematic review. Epidemiology and Psychiatric Sciences 2015;24(5):446‐57. [DOI] [PMC free article] [PubMed] [Google Scholar]
Nuechterlein KH, Subotnik KL, Turner LR, Ventura J, Becker DR, Drake RE. Individual placement and support for individuals with recent‐onset schizophrenia: Integrating supported education and supported employment. Psychiatric Rehabilitation Journal 2008;31(4):340‐9. [DOI] [PubMed] [Google Scholar]
Nuechterlein KH, Subotnik KL, Turner LR, Ventura J, Gitlin MJ, Gretchen‐Doorly D, et al. Individual placement and support after an initial episode of schizophrenia: impact on school or work recovery, hospitalization and utilization of disability support. Early Intervention in Psychiatry. 2012; Vol. 6, issue Suppl. 1:14.

O'Brien 2003 {published data only}

O’Brien A, Price C, Burns T, Perkins C. Improving the vocational status of patients with long‐term mental illness: a randomised controlled trial of staff training. Community Mental Health Journal 2003;39(4):333‐47. [DOI] [PubMed] [Google Scholar]

Oshima 2014 {published data only}

Oshima I, Sono T, Bond GR, Nishio M, Ito J. A randomized controlled trial of individual placement and support in Japan. Psychiatric Rehabilitation Journal 2014;37(2):137‐43. [DOI] [PubMed] [Google Scholar]

Penk 2010 {published data only}

Penk W, Drebing CE, Rosenheck RA, Krebs C, Ormer A, Mueller L. Veterans Health Administration transitional work experience vs. job placement in veterans with co‐morbid substance use and non‐psychotic psychiatric disorders. Psychatric Rehabilitation Journal 2010;33(4):297‐307. [DOI] [PubMed] [Google Scholar]

Schonebaum 2006 {published data only}

Gold PB, Macias C, Rodican CF. Does competitive work improve quality of life for adults with severe mental illness? Evidence from a randomized trial of supported employment. The Journal of Behavioral Health Services & Research 2016;43(2):155‐71. [DOI] [PMC free article] [PubMed] [Google Scholar]
Macias C, DeCarlo LT, Wang Q, Frey J, Barreira P. Work interest as a predictor of competitive employment: policy implications for psychiatric rehabilitation. Administration and Policy in Mental Health 2001;28(4):279‐97. [DOI] [PMC free article] [PubMed] [Google Scholar]
Macias C, Rodican CF, Hargreaves WA, Jones DR, Barreira PJ, Wang Q. Supported employment outcomes of a randomized controlled trial of ACT and Clubhouse models. Psychiatric Services 2006;57(10):1406‐15. [DOI] [PMC free article] [PubMed] [Google Scholar]
Schonebaum AD, Boyd JK. Work‐ordered day as a catalyst of competitive employment success. Psychiatric Rehabilitation Journal 2012;35(5):391. [DOI] [PubMed] [Google Scholar]
Schonebaum AD, Boyd JK, Dudek KJ. A comparison of competitive employment outcomes for the Clubhouse and PACT models. Psychiatric Services 2006;57(10):1416‐20. [DOI] [PubMed] [Google Scholar]

Tsang 2001 {published data only}

Tsang HWH. Rehab rounds: social skills training to help mentally ill persons find and keep a job. Psychiatric Services 2001;52(7):891‐4. [DOI] [PubMed] [Google Scholar]
Tsang HWH, Pearson V. Work‐related social skills training for people with schizophrenia in Hong Kong. Schizophrenia Bulletin 2001;27(1):139‐48. [DOI] [PubMed] [Google Scholar]

Tsang 2010 {published data only}

Tsang HWH. Supported employment versus traditional vocational rehabilitation for individuals with severe mental illness: a three‐year study. Hong Kong Medical Journal 2011;17(Suppl 2):S13‐7. [PubMed] [Google Scholar]
Tsang HWH, Chan A, Wong A, Liberman RP. Vocational outcomes of an integrated supported employment program for individuals with persistent and severe mental illness. Journal of Behavior Therapy and Experimental Psychiatry 2009;40(2):292‐305. [DOI] [PubMed] [Google Scholar]
Tsang HWH, Fung KM, Leung AY, Li SM, Cheung WM. Three year follow‐up study of an integrated supported employment for individuals with severe mental illness. Australian and New Zealand Journal of Psychiatry 2010;44(1):49‐58. [DOI] [PubMed] [Google Scholar]

Twamley 2012a {published data only}

Twamley EW, Narvaez JM, Becker DR, Bartels SJ, Jeste DV. Supported employment for middle‐aged and older people with schizophrenia. American Journal of Psychiatric Rehabilitation 2008;11:76‐89. [DOI] [PMC free article] [PubMed] [Google Scholar]
Twamley EW, Vella L, Burton CZ, Becker DR, Bell MD, Jeste DV. The efficacy of supported employment for middle‐aged and older people with schizophrenia. Schizophrenia Research 2012;135:100‐4. [DOI] [PMC free article] [PubMed] [Google Scholar]

Viering 2015 {published data only}

Landolt K, Brantchen E, Nordt C, Bärtch B, Kawohl W, Rössler W. Associations of supported employment with cognitive functioning and employment outcomes. Psychiatric Services 2016;67(11):1257‐61. [DOI] [PubMed] [Google Scholar]
Viering S, Bärtsch B, Obermann C, Rüsch N, Rössler W, Kawohl W. The effectiveness of individual placement and support for people with mental illness new on social benefits: a study protocol. BMC Psychiatry 2013;13:195. [DOI] [PMC free article] [PubMed] [Google Scholar]
Viering S, Jäger M, Bärtsch B, Nordt C, Rössler W, Warnke I, et al. Supported employment for the reintegration of disability pensioners with mental illnesses: a randomized controlled trial. Frontiers in Public Health 2015;3(237):1‐7. [DOI] [PMC free article] [PubMed] [Google Scholar]

Waghorn 2014 {published data only}

Waghorn G, Childs S, Hampton E, Gladman B, Greaves A, Bowman D. Enhancing community mental health services through formal partnerships with supported employment providers. American Journal of Psychiatric Rehabilitation 2012;15:157‐80. [Google Scholar]
Waghorn G, Dias S, Gladman B, Harris M, Saha S. A multi‐site randomised controlled trial of evidence‐based supported employment for adults with severe and persistent mental illness. Australian Occupational Therapy Journal 2014;61:424‐36. [DOI] [PubMed] [Google Scholar]

Walker 1969 {published data only}

Walker R, Winick W, Frost ES, Lieberman JM. Social restoration of hospitalised psychiatric patients through a program of special employment in industry. Rehabilitation Literature 1969;30(10):297‐303. [PubMed] [Google Scholar]

Wong 2008 {published data only}

Wong K, Chiu R, Tang B, Mak D, Liu J, Chiu SN. A randomized controlled trial of a supported employment program for persons with long‐term mental illness in Hong Kong. Psychiatric Services 2008;59(1):84‐90. [DOI] [PubMed] [Google Scholar]

Xiang 2007 {published data only}

Xiang YT, Weng YZ, Li WY, Gao L, Chen GL, Xie L, et al. Efficacy of the Community Re‐Entry Module for patients with schizophrenia in Beijing, China: outcome at 2‐year follow‐up. British Journal of Psychiatry 2007;190:49‐56. [DOI] [PubMed] [Google Scholar]

References to studies excluded from this review

Audini 1994 {published data only}

Audini B, Marks IM, Lawrence, RE, Connolly J, Watts V. Home‐based versus out‐patient/in‐patient care for people with serious mental illness. Phase II of a controlled study. The British Journal of Psychiatry 1994;165(2):204‐10. [DOI] [PubMed] [Google Scholar]
Knapp M, Marks I, Wolstenholme J, Beecham J, Astin, J, Audini B, et al. Home‐based versus hospital‐based care for serious mental illness. Controlled cost‐effectiveness study over four years. The British Journal of Psychiatry 1998;172(6):506‐12. [DOI] [PubMed] [Google Scholar]

Bateman 1999 {published data only}

Bateman A, Fonagy P. 8‐year follow‐up of patients treated for borderline personality disorder: mentalization‐based treatment versus treatment as usual. The American Journal of Psychiatry 2008;165(5):631‐8. [DOI] [PubMed] [Google Scholar]
Bateman A, Fonagy P. Effectiveness of partial hospitalization in the treatment of borderline personality disorder: a randomized controlled trial. American Journal of Psychiatry 1999;156:1563‐9. [DOI] [PubMed] [Google Scholar]
Bateman A, Fonagy P. Treatment of borderline personality disorder with psychoanalytically oriented partial hospitalization: an 18‐month follow‐up. American Journal of Psychiatry 2001;158:36‐42. [DOI] [PubMed] [Google Scholar]

Bayer 2008 {published data only}

Bayer W, Köster M, Salize HJ, Höhl W, Machleidt W, Wiedl KH, et al. Longer‐term effects of inpatient vocational and ergotherapeutic measures on the vocational integration of patients with schizophrenia [Längerfristige Auswirkungen stationärer arbeits‐ und ergotherapeutischer Maßnahmen auf die berufliche Integration schizophrener Patienten]. Psychiatrische Praxis 2008;35(4):170‐4. [DOI] [PubMed] [Google Scholar]

Becker 2007 {published data only}

Becker D, Whitley R, Bailey EL, Drake RE. Long‐term employment trajectories among participants with severe mental illness in supported employment. Psychiatric Services 2007;58(7):922‐8. [DOI] [PubMed] [Google Scholar]

Bell 1996 {published data only}

Bell MD, Lysaker PH. Clinical benefits of paid work activity in schizophrenia: 1‐year follow up. Schizophrenia Bulletin 1997;23(2):317‐28. [DOI] [PubMed] [Google Scholar]
Bell MD, Lysaker PH, Milstein RM. Clinical benefits of paid work activity in schizophrenia. Schizophrenia Bulletin 1996;22(1):51‐67. [DOI] [PubMed] [Google Scholar]
Bell MD, Milstein RM, Lysaker PH. Pay and participation in work activity: clinical benefits for clients with schizophrenia. Psychosocial Rehabilitation Journal 1993;17(2):173‐7. [Google Scholar]
Bell MD, Milstein RM, Lysaker PH. Pay as an incentive in work participation by patients with severe mental illness. Psychiatric Services 1993;44(7):684‐6. [DOI] [PubMed] [Google Scholar]

Bell 2003 {published data only}

Bell M, Lysaker P, Bryson G. A behavioral intervention to improve work performance in schizophrenia: work behavior inventory feedback. Journal of Vocational Rehabilitation 2003;1:43‐50. [Google Scholar]

Bell 2005 {published data only}

Bell MD, Bryson G, Greig T, Corcoran C, Wexler BE. Neurocognitive enhancement therapy with work therapy: effects on neuropsychological test performance. Archives of General Psychiatry 2001;8:763‐8. [DOI] [PubMed] [Google Scholar]
Bell MD, Bryson G, Wexler BE. Cognitive remediation of working memory deficits: durability of training effects in severely impaired and less severely impaired schizophrenia. Acta Psychiatrica Scandinavica 2003;2:101‐9. [DOI] [PubMed] [Google Scholar]
Bell MD, Bryson GJ, Greig TC, Fiszdon JM, Wexler BE. Neurocognitive enhancement therapy with work therapy: productivity outcomes at 6‐ and 12‐month follow‐ups. Journal of Rehabilitation Research and Development 2005;6:829‐38. [DOI] [PubMed] [Google Scholar]
Bell MD, Fiszdon J, Greig T, Wexler BE, Bryson G. Neurocognitive enhancement therapy with work therapy in schizophrenia: 6‐month follow‐up of neuropsychological performance. Journal of Rehabilitation Research and Development 2007;5:761‐70. [DOI] [PubMed] [Google Scholar]
Bell MD, Tsang HW, Greig TC, Bryson GJ. Neurocognition, social cognition, perceived social discomfort, and vocational outcomes in schizophrenia. Schizophrenia Bulletin 2009;35(4):738‐47. [DOI] [PMC free article] [PubMed] [Google Scholar]
Fiszdon JM, Bell MD. Cognitive remediation and work therapy in the outpatient treatment of patients with schizophrenia [Remédiation cognitive et thérapie occupationnelle dans le traitement ambulatoire du patient souffrant de schizophrénie]. Santé Mentale au Québec 2004;29(2):117‐42. [DOI] [PubMed] [Google Scholar]

Bell 2008b {published data only}

Bell MD, Zito W, Greig T, Wexler BE. Neurocognitive enhancement therapy with vocational services: work outcomes at two‐year follow‐up. Schizophrenia Research 2008;1‐3:18‐29. [DOI] [PubMed] [Google Scholar]
Bell MD, Zito W, GreigT, Wexler BE. Neurocognitive enhancement therapy and competitive employment in schizophrenia: effects on clients with poor community functioning. American Journal of Psychiatric Rehabilitation 2008;11(2):109‐22. [Google Scholar]
Greig TC, Zito W, Wexler BE, Fiszdon J, Bell MD. Improved cognitive function in schizophrenia after one year of cognitive training and vocational services. Schizophrenia Research 2007;96(1‐3):156‐61. [DOI] [PMC free article] [PubMed] [Google Scholar]

Bell 2014 {published data only}

Bell MD, Choi KH, Dyer C, Wexler BE. Benefits of cognitive remediation and supported employment for schizophrenia patients with poor community functioning. Psychiatric Services 2014;65(4):469‐75. [DOI] [PubMed] [Google Scholar]
Bell MD, Corbera S, Wexler BE. Cognitive remediation and competitive employment: differential benefits for schizophrenia patients with poor community function. Schizophrenia Research 2012;136:S78. [Google Scholar]
Bell MD, Corbera S, Wexler BE. Cognitive remediation and supported employment: moderators and mediators of vocational outcomes. Schizophrenia Bulletin 2013;39:S281‐S282. [Google Scholar]

Bertelsen 2008 {published data only}

Bertelsen M, Jeppesen P, Petersen L, Thorup A, Øhlenschlaeger J, Quach P, Christensen TØ, Krarup G, et al. First episode of psychosis intensive early intervention programme versus standard treatment‐‐secondary publication. Ugeskrift for Laeger 2009;171(41):2992‐5. [PubMed] [Google Scholar]
Bertelsen M, Jeppesen P, Petersen L, Thorup A, Øhlenschlaeger J, Quach P, et al. Five‐year follow‐up of a randomized multicenter trial of intensive early intervention vs standard treatment for patients with a first episode of psychotic illness: the OPUS trial. Archives of General Psychiatry 2008;65(7):762‐71. [DOI] [PubMed] [Google Scholar]
Bonebrake K, Bertelsen M, Thoru A, Pi Jeppesen P, Albert N, Krarup G, et al. Use of a latent variable framework to examine the relationship between symptom presentation at baseline and recovery at 5 years. Early Intervention in Psychiatry 2010;4:55. [Google Scholar]
Hastrup LH, Kronborg C, Bertelsen M, Jeppesen P, Jorgensen P, Petersen L, et al. Cost‐effectiveness of early intervention in first‐episode psychosis: economic evaluation of a randomised controlled trial (the OPUS study). British Journal of Psychiatry 2013;1:35‐41. [DOI] [PubMed] [Google Scholar]
Jørgensen P, Nordentoft M, Abel MB, Gouliaev G, Jeppesen P, Kassow P. Early detection and assertive community treatment of young psychotics: the Opus study rationale and design of the trial. Social Psychiatry and Psychiatric Epidemiology 2000;35(7):283‐7. [DOI] [PubMed] [Google Scholar]
Nordentoft M, Bertelsen M, Albert N, Jeppesen P, Petersen L, Thorup A, et al. The OPUS trial: a randomized multicentre single‐blinded trial of specialized assertive early intervention (OPUS Treatment) versus standard treatment for patients with a first episode of psychotic illness ‐ five‐year follow‐up. Early Intervention in Psychiatry 2010;4:24. [Google Scholar]
Nordentoft M, Melau M, Iversen T, Petersen L, Jeppesen P, Thorup A, et al. From research to practice: how OPUS treatment was accepted and implemented throughout Denmark. Early Intervention in Psychiatry 2015;9(2):156‐62. [DOI] [PubMed] [Google Scholar]
Thorup A, Albert N, Bertelsen M, Petersen L, Jeppesen P, Quack P, et al. Gender differences in first‐episode psychosis at 5‐year follow‐up – two different courses of disease? Results from the OPUS study at 5‐year follow‐up. European Psychiatry 2015;29(1):44‐51. [DOI] [PubMed] [Google Scholar]

Bond 2016 {published data only}

Bond GR, Drake RE, Campbell K. Effectiveness of individual placement and support supported employment for young adults. Early Intervention in Psychiatry 2016;10(4):300‐7. [DOI] [PMC free article] [PubMed] [Google Scholar]

Cook 2005 {published data only}

Burke‐Miller JK, Cook JA, Grey DD, Razzano LA, Blyler CR, Leff HS, et al. Demographic characteristics and employment among people with severe mental illness in a multisite study. Community Mental Health Journal 2006;42(2):143‐59. [DOI] [PubMed] [Google Scholar]
Cook JA, Blyler CR, Burke‐Miller JK, McFarlane WR, Leff HS, Mueser KT, et al. Effectiveness of supported employment for individuals with schizophrenia: results of a multi‐site, randomized trial. Clinical Schizophrenia & Related Psychoses 2008;1:37‐46. [Google Scholar]
Cook JA, Blyler CR, Leff HS, McFarlane WR, Goldberg RW, Gold PB, et al. The employment intervention demonstration program: major findings and policy implications. Psychiatric Rehabilitation Journal 2008;31(4):291‐5. [DOI] [PubMed] [Google Scholar]
Cook JA, Carey MA, Razzano LA, Burke J, Blyler CR. The pioneer: the employment intervention demonstration program. New Directions for Evaluation 2002;94:31‐44. [Google Scholar]
Cook JA, Leff HS, Blyler CR, Gold PB, Goldberg RW, Mueser KT, et al. Results of a multi site randomized trial of supported employment interventions for individuals with severe mental illness. Archives of General Psychiatry 2005;5:505‐12. [DOI] [PubMed] [Google Scholar]
Cook JA, Lehman AF, Drake R, McFarlane WR, Gold PB, Leff HS, et al. Integration of psychiatric and vocational services: a multisite randomized controlled trial of supported employment. American Journal of Psychiatry 2005;10:1948‐56. [DOI] [PubMed] [Google Scholar]
Cook JA, Mulkern V, Grey DD, Burke‐Miller J, Blyler CR, Razzano LA, et al. Effects of local unemployment rate on vocational outcomes in a randomized trial of supported employment for individuals with psychiatric disabilities. Journal of Vocational Rehabilitation 2006;25(2):71‐84. [Google Scholar]
Cook JA, Razzano LA, Burke‐Miller JK, Blyler CR, Leff HS, Mueser KT, et al. Effects of co‐occurring disorders on employment outcomes in a multisite randomized study of supported employment for people with severe mental illness. Journal of Rehabilitation Research and Development 2007;6:837‐49. [DOI] [PubMed] [Google Scholar]
Razzano LA Cook JA, Burke‐Miller JK, Mueser KT, Pickett‐Schenk SA, Grey DD, et al. Clinical factors associated with employment among people with severe mental illness: findings from the employment intervention demonstration program. Journal of Nervous and Mental Disease 2005;193(11):705‐13. [DOI] [PubMed] [Google Scholar]

Cook 2009 {published data only}

Cook S, Chambers E, Coleman JH. Occupational therapy for people with psychotic conditions in community settings: a pilot randomized controlled trial. Clinical Rehabilitation 2009;23(1):40‐52. [DOI] [PubMed] [Google Scholar]

Davis 2012 {published data only}

Davis LL, Leon AC, Toscano R, Drebing CE, Ward LC, Parker PE, et al. A randomized controlled trial of supported employment among veterans with posttraumatic stress disorder. Psychiatric Services 2012;63(5):464‐70. [DOI] [PubMed] [Google Scholar]
Davis LL, Pilkinton P, Poddar S, Blansett C, Toscano R, Parker PE. Impact of social challenges on gaining employment for veterans with posttraumatic stress disorder: an exploratory moderator analysis. Psychiatric Rehabilitation Journal 2014;37(2):107‐9. [DOI] [PubMed] [Google Scholar]

Davis 2015 {published data only}

Davis LW, Lysaker PH, Eicher AC. Effects of a mindfulness intervention on work outcomes for adults with schizophrenia. Schizophrenia Bulletin 2011;37:262‐3. [Google Scholar]
Davis LW, Lysaker PH, Kristeller JL, Salyers MP, Kovach AC, Woller S. Effect of mindfulness on vocational rehabilitation outcomes in stable phase schizophrenia. Psychological Services 2015;12(3):303‐12. [DOI] [PubMed] [Google Scholar]

Fowler 2009 {published data only}

Fowler D, Hodgekins J, Painter M, Reilly T, Crane C, Macmillan I, et al. Cognitive behaviour therapy for improving social recovery in psychosis: a report from the ISREP MRC Trial Platform study (Improving Social Recovery in Early Psychosis). Psychological Medicine 2009;39:1627‐36. [DOI] [PubMed] [Google Scholar]

Granholm 2014 {published data only}

Granholm E, Holden J, Link PC, McQuaid JR. Randomized clinical trial of cognitive behavioral social skills training for schizophrenia: improvement in functioning and experiential negative symptoms. Journal of Consulting and Clinical Psychology 2014;82(6):1173‐85. [DOI] [PMC free article] [PubMed] [Google Scholar]

Griffiths 1974 {published data only}

Griffiths RD. Rehabilitation of chronic psychotic patients. An assessment of their psychological handicap, an evaluation of the effectiveness of rehabilitation, and observations of the factors which predict outcome. Psycholical Medicine 1974;4(3):316‐25. [PubMed] [Google Scholar]

Hamilton 2000 {published data only}

Hamilton SH, Edgell ET, Revicki DA, Breier A. Functional outcomes in schizophrenia: a comparison of olanzapine and haloperidol in a European sample. International Clinical Psychopharmacology 2000;15(5):245‐55. [DOI] [PubMed] [Google Scholar]

Hasslet 2014 {published data only}

Haslett WR, McHugo GJ, Bond GR, Drake RE. Use of software for tablet computers to promote engagement with supported employment: results from an RCT. Psychiatric Services 2014;65(7):954‐6. [DOI] [PubMed] [Google Scholar]

Hirschfeld 2002 {published data only}

Hirschfeld RM, Dunner DL, Keitner G, Klein DN, Koran LM, Kornstein SG, et al. Does psychosocial functioning improve independent of depressive symptoms? A comparison of nefazodone, psychotherapy, and their combination. Biological Psychiatry 2002;51(2):123‐33. [DOI] [PubMed] [Google Scholar]

Hogarty 2004 {published data only}

Hogarty GE, Flesher S, Ulrich R, Carter M, Greenwald D, Pogue‐Geile M, et al. Archives of General Psychiatry. Cognitive enhancement therapy for schizophrenia: effects of a 2‐year randomized trial on cognition and behavior 2004;61(9):866‐76. [DOI] [PubMed] [Google Scholar]

Jager 2013 {published data only}

Jager M, Paras S, Nordt C, Warnke I, Bartsch B, Rossler W, Kawohl W. How sustainable is supported employment? A follow‐up investigation. Neuropsychiatrie 2013;27(4):196‐201. [DOI] [PubMed] [Google Scholar]

Kidd 2014 {published data only}

Kidd SA, Kaur J, Virdee G, George TP, McKenzie K, Herman Y. Cognitive remediation for individuals with psychosis in a supported education setting: a randomized controlled trial. Schizophrenia Research 2014;157(1):90‐8. [DOI] [PubMed] [Google Scholar]

Kline 1981 {published data only}

Kline MN, Hoisington V. Placing the psychiatrically disabled: a look at work values. Rehabilitation Counseling Bulletin 1981;24(5):366‐9. [Google Scholar]

Kopelowicz 1998 {published data only}

Kopelowicz A. Adapting social skills training for Latinos with schizophrenia. International Review of Psychiatry 1998;10:47‐50. [Google Scholar]

Kuldau 1977 {published data only}

Kuldau JM, Dirks SJ. Controlled evaluation of a hospital‐originated community transitional system. Archives of General Psychiatry 1977;34(11):1331‐40. [DOI] [PubMed] [Google Scholar]

Kurtz 2013 {published data only}

Kurtz MM, Mueser KT, Wexler B. Cognitive remediation and social skills training for schizophrenia. Schizophrenia Bulletin 2013;39:S339‐S340. [Google Scholar]

Kurtz 2015 {published data only}

Kurtz MM, Mueser KT, Thime WR, Corbera S, Wexler BE. Social skills training and computer‐assisted cognitive remediation in schizophrenia. Schizophrenia Research 2015;162(1‐3):35‐41. [DOI] [PMC free article] [PubMed] [Google Scholar]

Liberman 1998 {published data only}

Liberman RP, Wallace CJ, Blackwell G, Kopelowicz A, Vaccaro JV, Mintz J. Skills training versus psychosocial occupational therapy for persons with persistent schizophrenia. American Journal of Psychiatry 1998;155:1087‐91. [DOI] [PubMed] [Google Scholar]

Lindenmayer 2008 {published data only}

Lindenmayer JP, McGurk SR, Mueser KT, Khan A, Wance D, Hoffman L, et al. Randomized controlled trial of cognitive remediation among inpatients with persistent mental illness. Psychiatric Services 2008;59(3):241‐7. [DOI] [PubMed] [Google Scholar]

Lucca 2004 {published data only}

Lucca AM, Henry AD, Banks S, Simon L, Page S. Evaluation of an Individual Placement and Support model (IPS) program. Psychiatric Rehabilitation Journal 2004;27(3):251‐7. [DOI] [PubMed] [Google Scholar]

Luo 1994 {published data only}

Luo K, Yu D. Enterprise‐based sheltered workshops in Nanjing. A new model for the community rehabilitation of mentally ill workers. British Journal of Psychiatry 1994;24:89‐95. [PubMed] [Google Scholar]

Lysaker 2005 {published data only}

Davis LW, Ringer JM, Strasburger AM, Lysaker PH. Participant evaluation of a CBT program for enhancing work function in schizophrenia. Psychiatric Rehabilitation Journal 2008;32(1):55‐8. [DOI] [PubMed] [Google Scholar]
Lysaker PH, Bond G, Davis LW, Bryson GJ, Bell MD. Enhanced cognitive‐behavioral therapy for vocational rehabilitation in schizophrenia: effects on hope and work. Journal of Rehabilitation Research and Development 2005;42(5):673‐82. [DOI] [PubMed] [Google Scholar]
Lysaker PH, Davis LW, Beattie N. Effects of cognitive behavioral therapy and vocational rehabilitation on metacognition and coping in schizophrenia. Journal of Contemporary Psychotherapy 2006;1:25‐30. [Google Scholar]
Lysaker PH, Davis LW, Beattie N. Effects of cognitive behavioral therapy and vocational rehabilitation on metacognition and coping in schizophrenia: Erratum. Journal of Contemporary Psychotherapy 2007;37(2):115. [Google Scholar]

Lysaker 2009 {published data only}

Kukla M, Davis LW, Lysaker PH. Cognitive behavioral therapy and work outcomes: correlates of treatment engagement and full and partial success in schizophrenia. Behavioural and Cognitive Psychotherapy 2014;42(5):577‐92. [DOI] [PubMed] [Google Scholar]
Lysaker PH, Davis LW, Bryson GJ, Bell MD. Effects of cognitive behavioral therapy on work outcomes in vocational rehabilitation for participants with schizophrenia spectrum disorders. Schizophrenia Research 2009;107(2‐3):186‐91. [DOI] [PubMed] [Google Scholar]
Lysaker PH, Dimaggio G, Carcione A, Procacci M, Buck KD, Davis LW, et al. Metacognition and schizophrenia: the capacity for self‐reflectivity as a predictor for prospective assessments of work performance over six months. Schizophrenia Research 2010;122(1‐3):124‐30. [DOI] [PubMed] [Google Scholar]

Man 2012 {published data only}

Man DW, Law KM, Chung RC. Cognitive training for Hong Kong Chinese with schizophrenia in vocational rehabilitation. Hong Kong Medical Journal 2012;18(Suppl 6):18‐22. [PubMed] [Google Scholar]

McFarlane 2015 {published data only}

Carrión RE, Cornblatt BA, Burton CZ, Tso IF, Auther AM, Adelsheim S, et al. Personalized prediction of psychosis: external validation of the NAPLS‐2 Psychosis Risk Calculator with the EDIPPP Project. American Journal of Psychiatry 2016;173(10):989‐96. [DOI] [PMC free article] [PubMed] [Google Scholar]
Lynch S, McFarlane WR, Joly B, Adelsheim S, Auther A, Cornblatt BA, et al. Early detection, intervention and prevention of psychosis program: community outreach and early identification at six U.S. sites. Psychiatric Services (Washington, D.C.) 67;5:510‐6. [DOI] [PubMed] [Google Scholar]
McFarlane W. Early detection and intervention for the prevention of psychosis (EDIPPP): a national multisite effectiveness trial of indicated prevention in the USA. Early Intervention in Psychiatry 2012;6:2. [Google Scholar]
McFarlane WL, Cook W, Downing D, Ruff A, Lynch S, Adelsheim S, et al. Early detection, intervention, and prevention of psychosis program: rationale, design, and sample description. Adolescent Psychiatry 2012;2(2):112‐24. [Google Scholar]
McFarlane WR, Levin B, Travis L, Lucas FL, Lynch S, Verdi M, et al. Clinical and functional outcomes after 2 years in the early detection and intervention for the prevention of psychosis multisite effectiveness trial. Schizophrenia Bulletin 2015;41(1):30‐43. [DOI] [PMC free article] [PubMed] [Google Scholar]
Tso IF, Taylor SF, Grove TB, Niendam T, Adelsheim S, Auther A, et al. Factor analysis of the Scale of Prodromal Symptoms: data from the Early Detection and Intervention for the Prevention of Psychosis Program. Early Intervention Psychiatry 2017;11(1):14‐22. [DOI] [PMC free article] [PubMed] [Google Scholar]

McGrew 2005 {published data only}

McGrew JH, Johannesen JK, Griss ME, Born DL, Katuin C. Performance‐based funding of supported employment: a multi‐site controlled trial. Journal of Vocational Rehabilitation 2005;2:81‐99. [Google Scholar]
McGrew JH, Johannesen JK, Griss ME, Born DL, Katuin CH. Performance‐based funding of supported employment for persons with severe mental illness: vocational rehabilitation and employment staff perspectives. Journal of Behavioral Health Services and Research 2007;34(1):1‐16. [DOI] [PubMed] [Google Scholar]

McGurk 2003 {published data only}

McGurk SR, Mueser KT. Cognitive and clinical predictors of work outcomes in clients with schizophrenia receiving supported employment services: 4‐year follow‐up. Administration and Policy in Mental Health and Mental Health Services Research 2006;33(5):598‐606. [DOI] [PubMed] [Google Scholar]
McGurk SR, Mueser KT, Harvey PD, LaPuglia R, Marder J. Cognitive and symptom predictors of work outcomes for clients with schizophrenia in supported employment. Psychiatric Services 2003;54:1129‐35. [DOI] [PubMed] [Google Scholar]

Mueser 2005 {published data only}

Mueser KT, Aalto S, Becker DR, Ogden JS, Wolfe JS, Schiavo D, Wallace CJ, Xie H. The effectiveness of skills training for improving outcomes in supported employment. Psychiatric Services 2005;56(10):1254‐60. [DOI] [PubMed] [Google Scholar]

Mueser 2011 {published data only}

Mueser KT, Campbell K, Drake RE. The effectiveness of supported employment in people with dual disorders. Journal of Dual Diagnosis 2011;7(1‐2):90‐102. [DOI] [PMC free article] [PubMed] [Google Scholar]

Okpaku 1997 {published data only}

Okpaku SO, Anderson KH, Sibulkin AE, Butler JS, Bickman L. The effectiveness of a multidisciplinary case management intervention on the employment of SSDI applicants and beneficiaries. Psychiatric Rehabilitation Journal 1997;20(3):34‐41. [Google Scholar]

Resnick 2008 {published data only}

Resnick SG, Rosenheck RA, Canive JM, Souza C, Stroup TS, McEvoy J, et al. Employment outcomes in a randomized trial of second‐generation antipsychotics and perphenazine in the treatment of individuals with schizophrenia. Journal of Behavioral Health Services & Research 2008;2:215‐25. [DOI] [PubMed] [Google Scholar]

Rinaldi 2010 {published data only}

Rinaldi M, Perkins R, McNeil K, Hickman N, Singh SP. The individual placement and support approach to vocational rehabilitation for young people with first episode psychosis in the UK. Journal of Mental Health 2010;19(6):483‐91. [DOI] [PubMed] [Google Scholar]

Roder 2002 {published data only}

Roder V, Brenner HD, Müller D, Lächler M, Zorn P, Reisch T, et al. Development of specific social skills training programmes for schizophrenia patients: Results of a multicentre study. Acta Psychiatrica Scandinavica 2002;105(5):363‐71. [DOI] [PubMed] [Google Scholar]

Rogers 2006 {published data only}

Rogers ES, Anthony WA, Lyass A, Penk WE. A randomized clinical trial of vocational rehabilitation for people with psychiatric disabilities. Counseling Bulletin 2006;49(3):143‐56. [Google Scholar]

Rosen 2014 {published data only}

Rosen MI, Ablondi K, Black AC, Mueller L, Serowik KL, Martino S, et al. Work outcomes after benefit counselling among veterans applying for service connection for a psychiatric condition. Psychiatric Services 2014;65(12):1426‐32. [DOI] [PMC free article] [PubMed] [Google Scholar]

Rus 2013 {published data only}

Rus‐Calafell M, Gutiérrez‐Maldonado J, Ortega‐Bravo M, Ribas‐Sabaté J, Caqueo‐Urízar A. A brief cognitive‐behavioural social skills training for stabilised outpatients with schizophrenia: a preliminary study. Schizophrenia Research 2013;143(2‐3):372‐36. [DOI] [PubMed] [Google Scholar]

Sato 2014 {published data only}

Sato S, Iwata K, Furukawa SI, Matsuda Y, Hatsuse N, Ikebuchi E. The effects of the combination of cognitive training and supported employment on improving clinical and working outcomes for people with schizophrenia in Japan. Clinical Practice and Epidemiology in Mental Health 2014;10:18‐27. [DOI] [PMC free article] [PubMed] [Google Scholar]

Shi 2002 {published data only}

Shi L, Namjoshi MA, Zhang F, Gandhi G, Edgell ET, Tohen M, et al. Open‐label olanzapine treatment in bipolar I disorder: clinical and work functional outcomes. International Clinical Psychopharmacology 2002;17(5):227‐37. [DOI] [PubMed] [Google Scholar]

Sungur 2011 {published data only}

Sungur M, Soygur H, Guner P, Ustun B, Cetin I, Falloon IR. Identifying an optimal treatment for schizophrenia: a 2‐year randomized controlled trial comparing integrated care to a high‐quality routine treatment. International Journal of Psychiatry in Clinical Practice 2011;15(2):118‐27. [DOI] [PubMed] [Google Scholar]

Swildens 2011 {published data only}

Swildens W, Busschbach JT, Michon H, Kroon H, Koeter MW, Wiersma D, et al. Effectively working on rehabilitation goals: 24‐month outcome of a randomized controlled trial of the Boston psychiatric rehabilitation approach. The Canadian Journal of Psychiatry 2011;56(12):751‐60. [DOI] [PubMed] [Google Scholar]

Thunissen 2008 {published data only}

Thunnissen M, Duivenvoorden H, Busschbach J, Hakkaart‐van Roijen L, Tilburg W, Verheul R, et al. A randomized clinical trial on the effectiveness of a reintegration training program versus booster sessions after short‐term inpatient psychotherapy. Journal of Personality Disorders 2008;22(5):483‐95. [DOI] [PubMed] [Google Scholar]

Torrent 2013 {published data only}

Torrent C, Bonnin CM, Martinez‐Aran A, Valle J, Amann BL, Gonzalez‐Pinto A, et al. Efficacy of functional remediation in bipolar disorder: a multicenter randomized controlled study. American Journal of Psychiatry 2013;170(8):852‐9. [DOI] [PubMed] [Google Scholar]

Trapp 2013 {published data only}

Trapp W, Landgrebe M, Hoesl K, Lautenbacher S, Gallhofer B, Günther W, et al. Cognitive remediation improves cognition and good cognitive performance increases time to relapse‐‐results of a 5 year catamnestic study in schizophrenia patients. BMC Psychiatry 2013;13:184. [DOI] [PMC free article] [PubMed] [Google Scholar]

Tsang 2013 {published data only}

Tsang MM, Man DW. A virtual reality‐based vocational training system (VRVTS) for people with schizophrenia in vocational rehabilitation. Schizophrenia Research 2013;144(1‐3):51‐62. [DOI] [PubMed] [Google Scholar]

Twamley 2005 {published data only}

Twamley EW, Padin DS, Bayne KS, Narvaez JM, Williams RE, Jeste DV. Work rehabilitation for middle‐aged and older people with schizophrenia: a comparison of three approaches. Journal of Nervous and Mental Disease 2005;193(9):596‐601. [DOI] [PubMed] [Google Scholar]

Twamley 2012b {published data only}

Twamley EW, Savla GN, Zurhellen CH, Heaton RK, Jeste DV. Development and pilot‐testing of a novel compensatory cognitive training intervention for people with psychosis. American Journal of Psychiatric Rehabilitation 2008;11:144‐63. [DOI] [PMC free article] [PubMed] [Google Scholar]
Twamley EW, Vella L, Burton CZ, Heaton RK, Jeste DV. Compensatory cognitive training for psychosis: effects in a randomized controlled trial. The Journal of Clinical Psychiatry 2012;73(9):1212‐19. [DOI] [PMC free article] [PubMed] [Google Scholar]

Vauth 2005 {published data only}

Vauth R, Corrigan, PW, Clauss M, Dietl M, Dreher‐Rudolph M, Stieglitz RD, et al. Cognitive strategies versus self‐management skills as adjunct to vocational rehabilitation. Schizophrenia Bulletin 2005;31(1):55‐66. [DOI] [PubMed] [Google Scholar]

Wolkon 1971 {published data only}

Wolkon GH, Karmen M, Tanaka MHT. Evaluation of a social rehabilitation program for recently released psychiatric patients. Community Mental Health Journal 1971;7(4):312‐22. [DOI] [PubMed] [Google Scholar]

Xiang 2006 {published data only}

Xiang Y, Weng Y, Li W, Gao L, Chen G, Xie L, et al. Training patients with schizophrenia with the community re‐entry module. Social Psychiatry and Psychiatric Epidemiology 2006;41(6):464‐9. [DOI] [PubMed] [Google Scholar]

References to studies awaiting assessment

Bejerholm 2017 {published data only}

Bejerholm U, Larsson ME, Johanson S. Supported employment adapted for people with affective disorders—a randomized controlled trial. Journal of Affective Disorders 2017;207:212‐20. [DOI] [PubMed] [Google Scholar]

Glynn 2017 {published data only}

Glynn SM, Marder SR, Noordsy DL, O'Keefe C, Becker DR, Drake RE, et al. An RCT evaluating the effects of skills training and medication type on work outcomes among patients with schizophrenia. Psychiatric Services 2017;68(3):271‐7. [DOI] [PubMed] [Google Scholar]

Kane 2015 {published data only}

Kane JM, Robinson DG, Schooler NR, Mueser KT, Penn DL, Rosenheck RA, et al. Comprehensive versus usual community care for first‐episode psychosis: 2‐year outcomes from the NIMH RAISE early treatment program. American Journal of Psychiatry 2015;173(4):362‐72. [DOI] [PMC free article] [PubMed] [Google Scholar]
Kane JM, Schooler NR, Marcy P, Correll CU, Brunette MF, Mueser KT, et al. The RAISE early treatment program for first‐episode psychosis: background, rationale, and study design. The Journal of Clinical Psychiatry 2015;76(3):240‐6. [DOI] [PMC free article] [PubMed] [Google Scholar]
Rosenheck R, Leslie D, Sint K, Lin H, Robinson DG, Schooler NR, et al. Cost‐effectiveness of comprehensive, integrated care for first episode psychosis in the NIMH RAISE early treatment program. Schizophrenia Bulletin 2016;42(4):896‐906. [DOI] [PMC free article] [PubMed] [Google Scholar]
Rosenheck R, Mueser KT, Sint K, Lin H, Lynde DW, Glynn SM, et al. Supported employment and education in comprehensive, integrated care for first episode psychosis: effects on work, school, and disability income. Schizophrenia Research 2017;182:1201‐128. [DOI] [PubMed] [Google Scholar]

McGurk 2015 {published data only}

McGurk SR, Mueser KT, Xie H, Welsh J, Kaiser S, Drake RE, et al. Cognitive enhancement treatment for people with mental illness who do not respond to supported employment: a randomized controlled trial. The American Journal of Psychiatry 2015;72(9):852‐61. [DOI: 10.1176/appi.ajp.2015.14030374] [DOI] [PubMed] [Google Scholar]

McGurk 2016 {published data only}

McGurk SR, Mueser KT, Xie H, Feldman K, Shaya Y, Klein L, et al. Cognitive remediation for vocational rehabilitation nonresponders. Schizophrenia Research 2016;175(1‐3):48‐56. [DOI] [PubMed] [Google Scholar]

Schneider 2016 {published data only}

Schneider J, Akhtar A, Boycott N, Guo B, Latimer E, Cao Z, et al. Individual placement and support versus individual placement and support enhanced with work‐focused cognitive behaviour therapy: feasibility study for a randomised controlled trial. British Journal of Occupational Therapy 2016;79(5):257‐69. [Google Scholar]

References to ongoing studies

Bell 2015 {published data only}

Bell MD. Cognitive training to enhance work program outcomes: preliminary findings. Schizophrenia Bulletin 2015;41:S302. [Google Scholar]

Bitter 2015 {published data only}

Bitter NA, Roeg DP, Nieuwenhuizen C, Weeghel J. Effectiveness of the Comprehensive Approach to Rehabilitation (CARe) methodology: design of a cluster randomized controlled trial. BMC Psychiatry 2015;15(1):165. [DOI] [PMC free article] [PubMed] [Google Scholar]

Christensen 2015 {published data only}

Christensen TN, Nielsen IG, Stenager E, Morthorst BR, Linschou J, Nordentoft M, et al. Individual placement and support supplemented with cognitive remediation and work‐related social skills training in Denmark: study protocol for a randomized controlled trial. Trials 2015;16:280. [DOI] [PMC free article] [PubMed] [Google Scholar]

Granholm 2015 {published data only}

Granholm E, Holden JL, Sommerfeld D, Rufener C, Perivoliotis D, Mueser K, et al. Enhancing assertive community treatment with cognitive behavioral social skills training for schizophrenia: study protocol for a randomized controlled trial. Trials 16;1:438. [DOI] [PMC free article] [PubMed] [Google Scholar]

Harris 2015 {published data only}

Harris AW, Kosic T, Walker C, Gye W, Redoblado‐Hodge A. Internet based cognitive remediation can assist people with severe mental illness to gain and retain employment‐the Cogrem study. Schizophrenia Bulletin 2015;41:S314. [Google Scholar]

Melau 2011 {published data only}

Melau M, Jeppesen P, Thorup A, Bertelsen M, Petersen L, Gluud C, et al. The effect of five years versus two years of specialised assertive intervention for first episode psychosis ‐ OPUS II: study protocol for a randomized controlled trial. Trials 2011;12(1):1. [DOI] [PMC free article] [PubMed] [Google Scholar]

Nordt 2012 {published data only}

Nordt C, Brantschen E, Kawohl W, Bärtsch B, Haker H, Rüsch N, et al. ‘Placement budgets’ for supported employment–improving competitive employment for people with mental illness: study protocol of a multicentre randomized controlled trial. BMC Psychiatry 2012;1:165. [DOI: 10.1186/1471-244X-12-165] [DOI] [PMC free article] [PubMed] [Google Scholar]

Sveinsdottir 2014 {published data only}

Sveinsdottir V, Løvvik C, Fyhn T, Monstad K, Ludvigsen, K, Øverland S, et al. Protocol for the effect evaluation of Individual Placement and Support (IPS): a randomized controlled multicenter trial of IPS versus treatment as usual for patients with moderate to severe mental illness in Norway. BMC Psychiatry 2014;1:307. [DOI] [PMC free article] [PubMed] [Google Scholar]

Additional references

34 C.F.R. § 363.3(b) 2015

34 C.F.R. § 363.3(b) 2015. What definitions apply?. retrieved data March 12, 2015 from www.ecfr.gov.

Almerie 2015

Almerie MQ, Okbar Al Marhi M, Jawoosh M, Alsabbagh M, Matar HE, Maayan N, et al. Social skills programmes for schizophrenia. Cochrane Database of Systematic Reviews 2015, Issue 6. [DOI: 10.1002/14651858.CD009006.pub2] [DOI] [PMC free article] [PubMed] [Google Scholar]

Arbesman 2011

Arbesman M, Logsdon DW. Occupational therapy interventions for employment and education for adults with serious mental illness: a systematic review. American Journal of Occupational Therapy 2011;65(3):238‐46. [DOI] [PubMed] [Google Scholar]

Beard 1982

Beard JH, Propst RN, Malamud TJ. The Fountain House model of psychiatric rehabilitation. Psychosocial Rehabilitation Journal 1982;5(1):47‐53. [Google Scholar]

Becker 1993

Becker DR, Drake RE. A working life: the individual placement and support (IPS) program. Concord NH: New Hampshire‐Dartmouth Psychiatric Research Center, 1993. [Google Scholar]

Becker 1998

Becker DR, Drake RE, Bond GR, Xie H, Dain BJ, Harrison K. Job terminations among persons with severe mental illness participating in supported employment. Community Mental Health Journal 1998;34(1):71‐82. [DOI] [PubMed] [Google Scholar]

Becker 1999

Becker MA, Diamond R. Wisconsin Quality of Life Index (W‐QLI): a multidimensional model for measuring quality of life. Journal of Clinical Psychiatry 1999;60(Suppl 3):29‐31. [PubMed] [Google Scholar]

Bell 2008a

Bell MD, Zito W, Greig T, Wexler BE. Neurocognitive enhancement therapy with vocational services: work outcomes at two‐year follow‐up. Schizophrenia Research 2008;105(1):18‐29. [DOI] [PubMed] [Google Scholar]

Bellack 1993

Bellack AS, Mueser KT. Psychosocial treatment for schizophrenia. Schizophrenia Bulletin 1993;19(2):317‐36. [DOI] [PubMed] [Google Scholar]

Berwick 1991

Berwick DM, Murphy JM, Goldman PA, Ware JE Jr, Barsky AJ, Weinstein MC. Performance of a five‐item mental health screening test. Medical Care 1991;29(2):169‐76. [DOI] [PubMed] [Google Scholar]

Bilby 1992

Bilby R. A response to the criticisms of transitional employment. Psychosocial Rehabilitation Journal 1992;16(2):69‐82. [Google Scholar]

Boardman 2003

Boardman J. Work, employment and psychiatric disabilities. Advances in Psychiatric Treatment 2003;9:327‐34. [Google Scholar]

Bond 1997

Bond GR, Becker DR, Drake RE, Vogler KM. A fidelity scale for the individual placement and support model of supported employment. Rehabilitation Counseling Bulletin 1997;40(4):265‐84. [Google Scholar]

Bond 1999

Bond GR, Drake RE, Becker DR, Mueser KT. Effectiveness of psychiatric rehabilitation approaches for employment of people with severe mental illness. Journal of Disability Policy Studies 1999;10(1):18‐52. [Google Scholar]

Bond 2001a

Bond GR, Resnick SG, Drake RE, Xie H, McHugo GJ, Bebout RR. Does competitive employment improve nonvocational outcomes for people with severe mental illness?. Journal of Consulting and Clinical Psychology 2001;69(3):489‐501. [DOI] [PubMed] [Google Scholar]

Bond 2001b

Bond GR, Drake RE, Mueser K, Latimer E. Assertive community treatment for people with severe mental illness. Disease Management and Health Outcomes 2001;9(3):141‐59. [Google Scholar]

Bond 2002

Bond GR, Campbell K, Ecans LJ, Gervey R, Pascaris A, Tice S, et al. A scale to measure quality of supported employment for persons with severe mental illness. Journal of Vocational Rehabilitation 2002;17(4):239‐50. [Google Scholar]

Bond 2004

Bond GR, Salyers MP. Prediction of outcome from the Dartmouth assertive community treatment fidelity scale. CNS Spectrums 2004;9(12):937‐42. [DOI] [PubMed] [Google Scholar]

Bond 2012a

Bond G, Drake RE. Making the case for IPS supported employment. Administration and Policy in Mental Health and Mental Health Services Research 2012;41(1):69‐73. [DOI] [PubMed] [Google Scholar]

Bond 2012b

Bond GR, Petersen AE, Becker DR, Drake RE. Validation of the revised individual placement and support fidelity scale (IPS‐25). Psychiatric Services 2012;63:758‐63. [the fidelity scale and manual are available at: www.dartmouth.edu/˜ips/page19/page19.html] [DOI] [PubMed] [Google Scholar]

Bond 2015a

Bond GR, Drake RE, Luciano A. Employment and educational outcomes in early intervention programmes for early psychosis: a systematic review. Epidemiology and Psychiatric Sciences 2015;24(5):446‐57. [DOI] [PMC free article] [PubMed] [Google Scholar]

Bouvet 2014

Bouvet C, Battin C, Le‐Roy‐Hatala C. The Clubhouse model for people with severe mental illnesses: literature review and French experiment [Le modèle Clubhouse pour les personnes souffrant de troubles psychiques: revue de littérature et expérience francaise]. L'Encéphale 2014;41(6):477‐86. [DOI] [PubMed] [Google Scholar]

Boycott 2012

Boycott N, Schneider J, McMurran M. Additional interventions to enhance the effectiveness of individual placement and support: a rapid evidence assessment. Rehabilitation Research and Practice 2012. [DOI: 10.1155/2012/382420] [DOI] [PMC free article] [PubMed]

Burns 2009

Burns T, Catty J, White S, Becker T, Koletsi M, Fioritti A, et al. The impact of supported employment and working on clinical and social functioning: results of an international study of individual placement and support. Schizophrenia Bulletin 2009;35(5):949‐58. [DOI] [PMC free article] [PubMed] [Google Scholar]

Carlier 2013

Carlier BE, Schuring M, Lötters FJ, Bakker B, Borgers N, Burdorf A. The influence of re‐employment on quality of life and self‐rated health, a longitudinal study among unemployed persons in the Netherlands. BMC Public Health 2013;13(1):503. [DOI] [PMC free article] [PubMed] [Google Scholar]

Chaimani 2013

Chaimani A, Higgins JP, Mavridis D, Spyridonos P, Salanti G. Graphical tools for network meta‐analysis in STATA. PloS one 2013;8(10):e76654. [DOI] [PMC free article] [PubMed] [Google Scholar]

Chan 2015

Chan JY, Hirai HW, Tsoi KK. Can computer‐assisted cognitive remediation improve employment and productivity outcomes of patients with severe mental illness? A meta‐analysis of prospective controlled trials. Journal of Psychiatric Research 2015;68:293‐300. [DOI] [PubMed] [Google Scholar]

Corbière 2009

Corbière M, Lecomte T. Vocational services offered to people with severe mental illness. Journal of Mental Health 2009;18(1):38‐50. [Google Scholar]

Corbière 2013

Corbière M, Negrini A, Dewa C. Mental health problems and mental disorders ‐ linked determinants to work participation and work functioning. In: Loisel P, Anema JR editor(s). Handbook of Work Disability: Prevention and Management. New York: Springer Science+Business Media, 2013:267‐88. [Google Scholar]

Corbière 2014

Corbière M, Brouwers E, Lanctôt N, Weeghel J. Employment specialist competencies for supported employment programs. Journal of Occupational Rehabilitation 2014;24(3):484‐97. [DOI] [PubMed] [Google Scholar]

Corbière 2016

Corbière M, Lecomte T, Reinharz D, Kirsch B, Goering P, Menear M, et al. Predictors of acquisition of competitive employment for people enrolled in supported employment programs. Journal of Nervous and Mental Disease In press. [DOI] [PubMed]

Corrigan 2001

Corrigan PW. Place‐then‐train: an alternative service paradigm for persons with psychiatric disabilities. Science and Practice 2001;18:334‐49. [Google Scholar]

Crowther 2001

Crowther R, Marshall M, Bond G, Huxley P. Vocational rehabilitation for people with severe mental illness. Cochrane Database of Systematic Reviews 2001, Issue 2. [DOI: 10.1002/14651858.CD003080] [DOI] [PMC free article] [PubMed] [Google Scholar]

Delespaul 2013

Delespaul PH. Consensus regarding the definition of persons with severe mental illness and the number of such persons in the Netherlands. Tijdschrift voor Psychiatrie 2013;55(6):427‐38. [PubMed] [Google Scholar]

Dieterich 2010

Dieterich M, Irving CB, Park B, Marshall M. Intensive case management for severe mental illness. Cochrane Database of Systematic Reviews 2010, Issue 10. [DOI: 10.1002/14651858.CD007906.pub2] [DOI] [PMC free article] [PubMed] [Google Scholar]

Dilk 1996

Dilk MN, Bond GR. Meta‐analytic evaluation of skills training research for individuals with severe mental illness. Journal of Consulting and Clinical Psychology 1996;64(6):1337–46. [DOI] [PubMed] [Google Scholar]

Drake 1999a

Drake RE, Becker DR, Clark RE, Mueser KT. Research on the individual placement and support model of supported employment. Psychiatric Quarterly 1999;70(4):289‐301. [DOI] [PubMed] [Google Scholar]

Drake 2012

Drake RE, Bond GR, Becker DR. Individual placement and support: an evidence‐based approach to supported employment. Oxford University Press, 2012. [Google Scholar]

Gilbert 2013

Gilbert E, Marwaha S, Milton A, Johnson S, Morant N, Parsons N, et al. Social firms as a means of vocational recovery for people with mental illness: a UK survey. BMC Health Servey Research 2013;13:270. [DOI] [PMC free article] [PubMed] [Google Scholar]

Gold 2014

Gold PB, Macias C, Rodican CF. Does competitive work improve quality of life for adults with severe mental illness? Evidence from a randomized trial of supported employment. Journal of Behavioral Health Services and Research 2014 Feb 7 [Epub ahead of print]. [DOI] [PMC free article] [PubMed]

GRADE Working Group 2013

GRADE Working Group. Handbook for grading the quality of evidence and the strength and recommendations using the GRADE approach. GRADE Working Group, 2013. [Google Scholar]

Hatfield 1992

Hatfield B, Huxley P, Mohamad H. Accommodation and employment: a survey into the circumstances and expressed needs of users of mental health services in a northern town. British Journal of Social Work 1992;22(1):61‐73. [Google Scholar]

Heffernan 2011

Heffernan J, Pilkington P. Supported employment for persons with mental illness: systematic review of the effectiveness of individual placement and support in the UK. Journal of Mental Health 2011;20(4):368‐80. [DOI] [PubMed] [Google Scholar]

Higgins 2003

Higgins JPT, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta‐analyses. BMJ 2003;327:557‐60. [DOI] [PMC free article] [PubMed] [Google Scholar]

Higgins 2011a

Higgins JPT, Altman DG, Sterne JAC (editors). Chapter 8: Assessing risk of bias in included studies. In: Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 (updated March 2011). The Cochrane Collaboration, 2011. Available from handbook.cochrane.org.

Higgins 2011b

Higgins JPT, Deeks JJ, Altman DG (editors). Chapter 16: Special topics in statistics. In: Higgins JPT, Green S (editors), Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 (updated March 2011). The Cochrane Collaboration, 2011. Available from handbook.cochrane.org.

Higgins 2012

Higgins JPT, Jackson D, Barrett JK, Lu G, Ades AE, White IR. Consistency and inconsistency in network meta‐analysis: concepts and models for multi‐arm studies. Research Synthesis Methods 2012;3(2):98‐110. [DOI] [PMC free article] [PubMed] [Google Scholar]

Hogarty 1995

Hogarty GE, McEvoy JP, Ulrich RF, DiBarry AL, Bartone P, Cooley S, et al. Pharmacotherapy of impaired affect in recovering schizophrenic patients. Archives of General Psychiatry 1995;52(1):29‐41. [DOI] [PubMed] [Google Scholar]

Hsu 2009

Hsu TH, Ososkie J, Huang YT. Challenges in transition from sheltered workshop to competitive employment: perspectives of Taiwan social enterprise transition specialists. The Journal of Rehabilitation 2009;75(4):19. [Google Scholar]

Kay 1987

Kay SR, Fiszbein A, Opfer LA. The positive and negative syndrome scale (PANSS) for schizophrenia. Schizophrenia Bulletin 1987;13(2):261. [DOI] [PubMed] [Google Scholar]

Keshavan 2014

Keshavan MS, Vonogradov S, Rumsey J, Sherrill J, Wagner A. Cognitive training in mental disorders: update and future directions. American Journal of Psychiatry 2014;171(5):510‐22. [DOI] [PMC free article] [PubMed] [Google Scholar]

Kinoshita 2013

Kinoshita Y, Furukawa TA, Kinoshita K, Honyashiki M, Omori IM, Marshall M, et al. Supported employment for adults with severe mental illness. Cochrane Database of Systematic Reviews 2013, Issue 9. [DOI: 10.1002/14651858.CD008297.pub2] [DOI] [PMC free article] [PubMed] [Google Scholar]

Kopelowicz 2006

Kopelowicz A, Liberman RP, Zarate R. Recent advances in social skills training for schizophrenia. Schizophrenia Bulletin 2006;32(1):S12‐23. [DOI] [PMC free article] [PubMed] [Google Scholar]

Krainski 2013

Krainski J. Sheltered Employment. Encyclopedia of Autism Spectrum Disorders. New York: Springer, 2013:2855‐6. [DOI: 10.1007/978-1-4419-1698-3_1803] [DOI] [Google Scholar]

Kurtz 2008

Kurtz MM, Mueser KT. A meta‐analysis of controlled research on social skills training for schizophrenia. Journal of Consulting and Clinical Psychology 2008;76(3):491‐504. [DOI] [PubMed] [Google Scholar]

Latimer 2005

Latimer E. Community‐based care for people with severe mental illness in Canada. International Journal of Law and Psychiatry 2005;28:561–73. [DOI] [PubMed] [Google Scholar]

Lau 2005

Lau AL, Cummins RA, McPherson W. An investigation into the cross‐cultural equivalence of the Personal Wellbeing Index. Social Indicators Research 2005;72(3):403‐30. [Google Scholar]

Lehman 1988

Lehman AF. A quality of life interview for the chronically mentally ill. Evaluation and Program Planning 1988;11(1):51‐62. [Google Scholar]

Lipman 1982

Lipman RS. Differentiating anxiety and depression in anxiety disorders: use of rating scales. Psychopharmacology Bulletin 1982;18(4):69. [PubMed] [Google Scholar]

Loveland 2007

Loveland D, Driscoll H, Boyle M. Enhancing supported employment services for individuals with a serious mental illness: a review of the literature. Journal of Vocational Rehabilitation 2007;27(3):177‐89. [Google Scholar]

Lötters 2013

Lötters F, Carlier B, Bakker B, Borgers N, Schuring M, Burdorf A. The influence of perceived health on labour participation among long term unemployed. Journal of Occupational Rehabilitation 2013;23(2):300‐8. [DOI] [PubMed] [Google Scholar]

Marshall 1998

Marshall M, Lockwood A. Assertive community treatment for people with severe mental disorders. Cochrane Database of Systematic Reviews 1998, Issue 2. [DOI: 10.1002/14651858.CD001089.pub2] [DOI] [PubMed] [Google Scholar]

Marwaha 2007

Marwaha S, Johnson S, Bebbington P, Stafford M, Angermeyer MC, Brugha T, et al. Rates and correlates of employment in people with schizophrenia in the UK, France and Germany. The British Journal of Psychiatry 2007;191:30‐7. [DOI] [PubMed] [Google Scholar]

McGurk 2004

McGurk SR, Mueser KT. Cognitive functioning, symptoms, and work in supported employment: a review and heuristic model. Schizophrenia Research 2004;70(2):147‐73. [DOI] [PubMed] [Google Scholar]

McGurk 2005

McGurk SR, Mueser KT, Pascaris A. Cognitive training and supported employment for persons with severe mental illness: one‐year results from a randomized controlled trial. Schizophrenia Bulletin 2005;31(4):898‐909. [DOI] [PubMed] [Google Scholar]

McGurk 2014

McGurk SR, Mueser KT. Cognitive remediation and supported employment. In: Bukley PF, Gaughran F editor(s). Treatment‐Refractory Schizophrenia ‐ a Clinical Conundrum. Berlin Heidelberg: Springer‐Verlag, 2014:174. [ISBN 978‐3‐64245256‐7] [Google Scholar]

McQuilken 2003

McQuilken M, Zahniser JH, Novak J, Starks, RD, Olmos A, et al. The work project survey: consumer perspectives on work. Journal of Vocational Rehabilitation 2003;18(1):59‐68. [Google Scholar]

Migliore 2010

Migliore A. Sheltered Workshops. In: JH Stone, M Blouin, editors. International Encyclopedia of Rehabilitation 2010. [http://cirrie.buffalo.edu/encyclopedia/en/article/136/]

Modini 2016

Modini M, Tan L, Brinchmann B, Wang MJ, Killackey E, Glozier N, et al. Supported employment for people with severe mental illness: systematic review and meta‐analysis of the international evidence. British Journal of Psychiatry 2016;209(1):14‐22. [DOI] [PubMed] [Google Scholar]

Mueser 2001

Mueser KT, Salyers MP, Mueser PR. A prospective analysis of work in schizophrenia. Schizophrenia Bulletin 2001;27(2):281‐96. [DOI] [PubMed] [Google Scholar]

Mueser 2014

Mueser KT, McGurk SR. Supported employment for persons with serious mental illness: current status and future directions. L'Encéphale 2014;40:S45‐S56. [DOI] [PubMed] [Google Scholar]

Norman 2006

Norman, C. The Fountain House movement, an alternative rehabilitation model for people with mental health problems, members’ descriptions of what works. Scandinavian Journal of Caring Sciences 2006;20(2):184‐92. [DOI] [PubMed] [Google Scholar]

OECD 2012

OECD. Sick on the job? Myths and realities about mental health and work. Mental Health and Work. OECD Publishing, 2012. [Google Scholar]

Olliver 1997

Oliver JPJ, Huxley PJ, Priebe S, Kaiser W. Measuring the quality of life of severely mentally ill people using the Lancashire Quality of Life Profile. Social psychiatry and psychiatric epidemiology 1997;32(2):76‐83. [DOI] [PubMed] [Google Scholar]

Overall 1962

Overall JE, Gorham DR. The Brief Psychiatric Rating Scale. Psychological reports 1962;10(3):799‐812. [Google Scholar]

Perkins 2009

Perkins D, Raines J, Tschopp M, Warner T. Gainful employment reduces stigma toward people recovering from schizophrenia. Community Mental Health Journal 2009;45(3):158‐62. [DOI] [PubMed] [Google Scholar]

Priebe 1999

Priebe S, Huxley P, Knight S, Evans S. Application and results of the Manchester Short Assessment of Quality of Life. International Journal of Social Psychiatry 1999;45:7‐12. [DOI] [PubMed] [Google Scholar]

Raskin 1969

Raskin A, Schulterbrandt J, Reatig N, McKeon JJ. Replication of factors of psychopathology in interview, ward behavior and self‐report ratings of hospitalized depressives. The Journal of Nervous and Mental Disease 1969;148(1):87‐98. [DOI] [PubMed] [Google Scholar]

RevMan 2014 [Computer program]

Nordic Cochrane Centre, The Cochrane Collaboration. Review Manager 5 (RevMan) 5. Version 5.3. Nordic Cochrane Centre, The Cochrane Collaboration, 2014.

Ruggeri 2000

Ruggeri M, Leese M, Thornicroft G, Bisoffi G, Tansella M. Definition and prevalence of severe and persistent mental illness. The British Journal of Psychiatry 2000;177(2):149‐55. [DOI] [PubMed] [Google Scholar]

Salanti 2011

Salanti G, Ades AE, Ioannidis JP. Graphical methods and numerical summaries for presenting results from multiple‐treatment meta‐analysis: an overview and tutorial. Journal of Clinical Epidemiology 2011;64(2):163‐71. [DOI] [PubMed] [Google Scholar]

Salanti 2014

Salanti G, Giovane C, Chaimani A, Caldwell DM, Higgins JP. Evaluating the quality of evidence from a network meta‐analysis. PloS One 2014;9(7):e99682. [DOI] [PMC free article] [PubMed] [Google Scholar]

Salkever 2007

Salkever DS, Karakus MC, Slade EP, Harding CM, Hough RL, Rosenheck RA, et al. Measures and predictors of community‐based employment and earnings of persons with schizophrenia in a multisite study. Psychiatric Services 2007;58(3):315‐24. [DOI] [PubMed] [Google Scholar]

Savio 1993

Savio M, Righetti A. Cooperatives as a social enterprise in Italy: a place for social integration and rehabilitation. Acta Psychiatrica Scandinavica 1993;88:238‐42. [DOI] [PubMed] [Google Scholar]

Schandelmaier 2012

Schandelmaier S, Ebrahim S, Burkhardt SC, Boer WE, Zumbrunn T, Guyatt, et al. Return to work coordination programmes for work disability: a meta‐analysis of randomised controlled trials. PLoS One 2012;7(11):e49760. [DOI] [PMC free article] [PubMed] [Google Scholar]

Schuring 2013

Schuring M, Robroek SJ, Otten FW, Arts CH, Burdorf A. The effect of ill health and socioeconomic status on labor force exit and re‐employment: a prospective study with ten years follow‐up in the Netherlands. Scandinavian Journal of Work, Environment and Health 2013;39(2):134‐43. [DOI] [PubMed] [Google Scholar]

Stein 1998

Stein LI, Alberto BS. Assertive Community Treatment of persons with severe mental illness. New York: WW Norton & Co, 1998. [Google Scholar]

Svanberg 2010

Svanberg E, Gumley A, Wilson A. How do social firms contribute to recovery from mental illness? A qualitative study. Clinical Psychology and Psychotherapy 2010;17(6):482–96. [DOI] [PubMed] [Google Scholar]

Tsang 2009

Tsang HW, Chan A, Wong A, Liberman RP. Vocational outcomes of an integrated supported employment program for individuals with persistent and severe mental illness. Journal of Behavior Therapy and Experimental Psychiatry 2009;40(2):292‐305. [DOI] [PubMed] [Google Scholar]

Twamley 2003

Twamley EW, Jeste DV, Lehman AF. Vocational rehabilitation in schizophrenia and other psychotic disorders: a literature review and meta‐analysis of randomized controlled trials. Journal of Nervous and Mental Disease 2003;191(8):515‐23. [DOI] [PubMed] [Google Scholar]

Vilotti 2014

Villotti P, Zaniboni S, Fraccaroli F. Social cooperatives in Italy [Les entreprises à économie sociale en Italie]. L'Encéphale 2014;40:S57‐S65. [DOI] [PubMed] [Google Scholar]

Ware 1996

Ware JE Jr, Kosinski M, Keller SD. A 12‐item short‐form health survey: construction of scales and preliminary tests of reliability and validity. Medical Care 1996;34(3):22‐233. [DOI] [PubMed] [Google Scholar]

White 2011

White IR. Multivariate random‐effects meta‐regression: updates to mvmeta. The Stata Journal 2011;11(2):255‐70. [Google Scholar]

White 2012

White IR, Barrett JK, Jackson D, Higgins JPT. Consistency and inconsistency in network meta‐analysis: model estimation using multivariate meta‐regression. Research Synthesis Methods 2012;3:111‐25. [DOI] [PMC free article] [PubMed] [Google Scholar]

Wing 1961

Wing JK. A simple and reliable subclassification of chronic schizophrenia. The British Journal of Psychiatry 1961;107(450):862‐75. [DOI] [PubMed] [Google Scholar]

Wykes 2011

Wykes T, Huddy V, Cellard C, McGurk SR, Czobor P. A meta‐analysis of cognitive remediation for schizophrenia: methodology and effect sizes. American Journal of Psychiatry 2011;168(5):472‐85. [DOI] [PubMed] [Google Scholar]

Zigmond 1983

Zigmond AS, Snaith RP. The hospital anxiety and depression scale. Acta Psychiatrica Scandinavica 1983;67(6):361‐70. [DOI] [PubMed] [Google Scholar]

References to other published versions of this review

Suijkerbuijk 2015

Suijkerbuijk YB, Verbeek JH, Mechelen J, Ojajärvi A, Anema JR, Corbiere M, et al. Interventions for obtaining and maintaining employment in adults with severe mental illness, a network meta‐analysis. Cochrane Database of Systematic Reviews 2015, Issue 9. [DOI: 10.1002/14651858.CD011867] [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD011867-bib-0001] Au DW, Tsang HW, So WW, Bell MD, Cheung V, Yiu MG, et al. Effects of integrated supported employment plus cognitive remediation training for people with schizophrenia and schizoaffective disorders. Schizophrenia Research 2015;166(1):297‐303. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0002] Beard JH, Malamud TJ, Rossman E. Psychiatric rehabilitation and long‐term rehospitalisation rates: the findings of two research studies. Schizophrenia Bulletin 1978;4:622‐35. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0003] Beard JH, Pitt MA, Fisher SH, Goertzel V. Evaluating the effectiveness of a psychiatric rehabilitation program. American Journal of Orthopsychiatry 1963;33:701‐12. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0004] Becker RE. An evaluation of a rehabilitation program for chronically hospitalised psychiatric patients. Social Psychiatry 1967;2:32‐8. [Google Scholar]

[CD011867-bib-0005] Areberg C, Bejerholm U. The effect of IPS on participants' engagement, quality of life, empowerment, and motivation: a randomized controlled trial. Scandinavian Journal of Occupational Therapy 2013;20(6):420‐8. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0006] Bejerholm U, Areberg C, Hofgren C, Sandlund M, Rinaldi M. Individual placement and support in Sweden ‐ a randomized controlled trial. Nordic Journal of Psychiatry 2015;69(1):57‐66. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0007] Blankertz L, Robinson S. Adding a vocational focus to mental health rehabilitation. Psychiatric Services 1996;47:1216‐22. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0008] Bond GR, Dincin J. Accelerating entry into transitional employment in a psychosocial rehabilitation agency. Rehabilitation Psychology 1986;31:143‐55. [Google Scholar]

[CD011867-bib-0009] Bond GR, Dietzen LL, McGrew JH, Miller LD. Accelerating entry into supported employment for persons with severe psychiatric disabilities. Rehabilitation Psychiatry 1995;40:75‐94. [Google Scholar]

[CD011867-bib-0010] Bond GR, Dietzen LL, Vogler K, Katuin CH, McGrew JH, Miller LD. Toward a framework for evaluating cost and benefits of psychiatric rehabilitation: three case examples. Journal of Vocational Rehabilitation 1995;5:75‐88. [Google Scholar]

[CD011867-bib-0011] Bond GR, Salyers MP, Dincin J, Drake R, Becker DR, Fraser VV, et al. A randomised controlled trial comparing two vocational models for persons with severe mental Illness. Journal of Consulting and Clinical Psychology 2007;75(6):968‐82. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0012] Fraser VV, Jones AM, Frounfelker R, Harding B, Hardin T, Bond GR. VR closure rates for two vocational models. Psychiatric Rehabilitation Journal 2008;31(4):332‐9. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0013] Harding B, Torres‐Harding S, Bond GR, Salyers MP, Rollins AL, Hardin T. Factors associated with early attrition from psychosocial rehabilitation programs. Community Mental Health Journal 2008;44(4):283‐8. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0014] Kukla M, Bond GR. A randomised controlled trial of evidence based supported employment: nonvocational outcomes. Journal of Vocational Rehabilitation 2013;38:91‐8. [Google Scholar]

[CD011867-bib-0015] Kukla M, Bond GR. The working alliance and employment outcomes for people with severe mental illness enrolled in vocational programs. Rehabilitation Psychology 2009;54(2):157‐63. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0016] Bond GR, Kim SJ, Becker DR, Swanson SJ, Drake RE, Krzos IM, et al. A controlled trial of supported employment for people with severe mental illness and justice involvement. Psychiatric Services 2015;66(10):1027‐34. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0019] Burns T, Catty J. IPS in Europe: the EQOLISE trial. Psychiatric rehabilitation journal 2008;31(4):313‐17. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0017] Burns T, Catty J, Becker T, Drake, RE, Fioritti A, Knapp M, et al. The effectiveness of supported employment for people with severe mental illness: a randomised controlled trial in six European countries. The Lancet 2007;370:1146–52. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0018] Burns T, Catty J, White SJ, Becker T, Koletsi M, Fioritti A, et al for the EQOLISE Group. The impact of supported employment and working on clinical and social functioning: results of an international study of individual placement and support. Schizophrenia Bulletin 2009;35(5):949–58. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD011867-bib-0020] Burns T, White SJ, Catty J, for the EQOLISE Group. Individual placement and support in Europe: the EQOLISE trial. International Review of Psychiatry 2008;20(6):498‐502. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0021] Catty J, Lissouba P, White SJ, Becker T, Drake RE, Fioritti A, et al on behalf of the EQOLISE Group. Predictors of employment for people with severe mental illness: results of an international six‐centre randomised controlled trial. The British Journal of Psychiatry 2008;192:224‐31. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0022] Catty J, White SJ, Koletsi M, Becker T, Fioritti A, Kalkan R, et al for the EQOLISE group. Therapeutic relationships in vocational rehabilitation: predicting good relationships for people with psychosis. Psychiatry Research 2011;187(1):68‐73. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0023] Kilian R, Lauber C, Kalkan R, Dorn W, Rössler W, Wiersma D, et al. The relationships between employment, clinical status, and psychiatric hospitalisation in patients with schizophrenia receiving either IPS or a conventional vocational rehabilitation programme. Social Psychiatry and Psychiatric Epidemiology 2012;47(9):1381‐9. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0024] Knapp M, Patel A, Curran C, Latimer E, Catty J, Becker T, et al. Supported employment: cost‐effectiveness across six European sites. World Psychiatry 2013;12(1):60‐8. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD011867-bib-0025] Koletsi M, Niersman A, Busschbach Van JT, Catty J, Becker T, et al for the EQOLISE Group. Working with mental health problems: clients’ experiences of IPS, vocational rehabilitation and employment. Social Psychiatry and Psychiatric Epidemiology 2009;44(11):961‐70. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0026] Burgess J, Yeeles K, Burns T. Refining Individual Placement and Support (IPS) and establishing its effectiveness: a pragmatic, non‐inferiority RCT (IPS‐LITE trial). Psychiatrische Praxis 2011;38:congress abstract. [Google Scholar]

[CD011867-bib-0027] Burns T, Yeeles K, Langford O, Montes MV, Burgess J, Anderson C. A randomised controlled trial of time‐limited individual placement and support: IPS‐LITE trial. The British Journal of Psychiatry 2015;207:351‐6. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0028] Chandler D, Hu TW, Meisel J, McGowen M, Madison K. Mental health costs, other public costs, and family burden among mental health clients in capitated integrated service agencies. Journal of Mental Health Administration 1997;24:178‐88. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0029] Chandler D, Meisel J, Hu TW, McGowen M, Madison K. A capitated model for a cross section of severely mentally ill clients: employment outcomes. Community Mental Health Journal 1997;33:501‐16. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0030] Chandler D, Meisel J, Hu TW, McGowen M, Madison K. Client outcomes in a three‐year controlled study of an integrated service agency model. Psychiatric Services 1996;47:1337‐43. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0031] Chandler D, Meisel J, Hu TW, Mcgowen M, Madison K. A capitated model for a cross‐section of severely mentally ill clients: hospitalization. Community Mental Health Journal 1998;34(1):13‐26. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0032] Chandler D, Meisel J, McGowen M, Mintz J, Madison K. Client outcomes in two model capitated integrated service agencies. Psychiatric Services 1996;47:175‐80. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0033] Craig T, Shepherd G, Rinaldi M, Smith J, Carr S, Preston F, et al. Vocational rehabilitation in early psychosis: cluster randomised trial. British Journal of Psychiatry 2014;205(2):145‐50. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0034] Bond GR. An economic analysis of psychosocial rehabilitation. Hospital and Community Psychiatry 1984;35:356‐62. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0035] Dincin J, Witheridge TF. Psychiatric rehabilitation as a deterrent to recidivism. Hospital and Community Psychiatry 1982;33:645‐50. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0036] Becker DR, Drake RE, Bond GR, Xie H, Dain BJ, Harrison K. Job terminations among persons with severe mental illness participating in supported employment. Community Mental Health Journal 1998;34(1):71‐82. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0037] Clark RE. Supported employment and managed care: can they coexist?. Psychiatric Rehabilitation Journal 1998;22(1):62‐68. [Google Scholar]

[CD011867-bib-0038] Drake RE, McHugo GJ, Becker DR, Anthony WA, Clark RE. The New Hampshire study of supported employment for people with severe mental illness. Journal of Consulting and Clinical Psychology 1996;64(2):391‐9. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0039] Mueser KT, Becker DR, Torrey WC, Xie H, Bond GR, Drake REJet al. Work and non‐vocational domains of functioning in persons with severe mental illness: a longitudinal analysis. Journal of Nervous and Mental Disease 1997;185(7):419‐26. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0040] Torrey WC, Mueser KT, McHugo GH, Drake RE. Self‐esteem as an outcome measure in studies of vocational rehabilitation for adults with severe mental illness. Psychiatric Services 2000;51(2):229‐33. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0041] Dixon L, Hoch JS, Clark R, Bebout R, Drake R, McHugo G, et al. Cost‐effectiveness of two vocational rehabilitation programs for persons with severe mental illness. Psychiatric Services 2002;53(9):1118‐24. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0042] Drake RE, McHugo GJ, Bebout RR, Becker DR, Harris, M, Bond GR, et al. A randomised clinical trial of supported employment for inner‐city patients with severe mental disorders. Archives of General Psychiatry 1999;56(7):627–33. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0043] Drake RE, Frey W, Bond GR, Goldman HH, Salkever D, Miller A, et al. Assisting Social Security Disability Insurance beneficiaries with schizophrenia, bipolar disorder, or major depression in returning to work. American Journal of Psychiatry 2013;170(12):1433‐41. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0044] Frey WD, Azrin ST, Goldman HH, Kalasunas S, Salkever DS, Miller AL, et al. The mental health treatment study. Psychiatric Rehabilitation Journal 2008;31(4):306‐12. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0045] Salkever DS, Gibbons B, Drake RE, Frey WD, Hale TW, Karakus M. Increasing earnings of Social Security Disability Income beneficiaries with serious mental disorder. The Journal of Mental Health Policy and Economics 2014;17(2):75‐90. [PubMed] [Google Scholar]

[CD011867-bib-0046] Drebing CE, Ormer EA, Krebs C, Rosenheck R, Rounsaville B, Herz L, et al. The impact of enhanced incentives on vocational rehabilitation outcomes for dually diagnosed veterans. Journal of Applied Behavioral Analysis 2005;38(3):359‐72. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD011867-bib-0047] Drebing CE, Ormer EA, Mueller L, Hebert M, Penk WE, Petry NM, et al. Adding contingency management intervention to vocational rehabilitation: outcomes for dually diagnosed veterans. Journal of Rehabilitation Research & Development 2007;44(6):851‐66. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0048] Eack SM, Greenwald DP, Hogarty SS, Cooley SJ, DiBarry AL, Montrose DM, et al. Cognitive enhancement therapy for early‐course schizophrenia: effects of a two‐year randomized controlled trial. Psychiatric Services 2009;60(11):1468‐76. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD011867-bib-0049] Eack SM, Greenwald DP, Hogarty SS, Keshavan MS. One‐year durability of the effects of cognitive enhancement therapy on functional outcome in early schizophrenia. Schizophrenia Research 2010;120(1):210‐16. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD011867-bib-0050] Eack SM, Hogarty GE, Cho RY, Prasad KM, Greenwald DP, Hogarty SS, et al. Neuroprotective effects of cognitive enhancement therapy against gray matter loss in early schizophrenia: results from a 2‐year randomized controlled trial. Archives of General Psychiatry 2010;67(7):674‐82. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD011867-bib-0051] Eack SM, Hogarty GE, Greenwald DP, Hogarty SS, Keshavan MS. Cognitive enhancement therapy improves emotional intelligence in early course schizophrenia: preliminary effects. Schizophrenia Research 2007;89(1):308‐11. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD011867-bib-0052] Eack SM, Hogarty GE, Greenwald DP, Hogarty SS, Keshavan MS. Effects of cognitive enhancement therapy on employment outcomes in early schizophrenia: results from a 2‐year randomized trial. Research on Social Work Practice 2011;21(1):32‐42. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD011867-bib-0053] Eack SM, Pogue‐Geile MF, Greenwald DP, Hogarty SS, Keshavan MS. Mechanisms of functional improvement in a 2‐year trial of cognitive enhancement therapy for early schizophrenia. Psychological Medicine 2011;41(6):1253‐61. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD011867-bib-0054] Keshavan MS, Eack SM, Wojtalik JA, Prasad KM, Francis AN, Bhojraj TS, et al. A broad cortical reserve accelerates response to cognitive enhancement therapy in early course schizophrenia. Schizophrenia Research 2011;130(1):123‐9. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD011867-bib-0055] Lewandowski KE, Eack SM, Hogarty SS, Greenwald DP, Keshavan MS. Is cognitive enhancement therapy equally effective for patients with schizophrenia and schizoaffective disorder?. Schizophrenia Research 2011;152(2):291‐4. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0056] Gervey R, Bedell JR. Supported employment in vocational rehabilitation. Psychological Assessment and Treatment of Persons with Severe Mental Disorders. Washington DC: Taylor & Francis, 1994:170‐5. [Google Scholar]

[CD011867-bib-0057] Gold PB, Meisler N, Santos AB, Carnemolla MA, Williams OH, Keleher J. Randomized trial of supported employment integrated with assertive community treatment for rural adults with severe mental illness. Schizophrenia Bulletin 2006;32(2):378‐95. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD011867-bib-0058] Hoffmann H, Jäckel D, Glauser S, Kupper Z. A randomised controlled trial of the efficacy of supported employment. Acta Psychiatrica Scandinavica 2012;125(2):157‐67. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0059] Hoffmann H, Jäckel D, Glauser S, Mueser KT, Kupper Z. Long‐term effectiveness of supported employment: 5‐year follow‐up of a randomized controlled trial. American Journal of Psychiatry 2014;171(11):1183‐90. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0060] Heslin M, Howard L, Leese M, McCrone P, Rice C, Jarrett M, et al. Randomized controlled trial of supported employment in England: 2 year follow‐up of the Supported Work And Needs (SWAN) study. World Psychiatry 2011;10(2):132‐7. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD011867-bib-0061] Howard LM, Heslin M, Leese M, McCrone P, Rice C, Jarrett M, et al. Supported employment: randomised controlled trial. British Journal of Psychiatry 2010;196(5):404‐11. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD011867-bib-0062] Baksheev GN, Allott K, Jackson HJ, McGorry PD, Killackey E. Predictors of vocational recovery among young people with first‐episode psychosis: findings from a randomized controlled trial. Psychiatric Rehabilitation Journal 2012;35(6):421‐7. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0063] Killackey E, Jackson HJ, McGorry PD. Vocational intervention in first‐episode psychosis: individual placement and support v. treatment as usual. British Journal of Psychiatry 2008;193(2):114‐20. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0064] Allott KA, Cotton SM, Chinnery GL, Baksheev GN, Massey J, Sun P, et al. The relative contribution of neurocognition and social cognition to 6‐month vocational outcomes following individual placement and support in first‐episode psychosis. Schizophrenia Research 2013;150(1):136‐43. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0065] Killackey E, Allot K, Cotton SM, Jackson H, Scutella R, Tseng YP, et al. A randomized controlled trial of vocational intervention for young people with first‐episode psychosis: method. Early Intervention in Psychiatry 2013;7:329‐37. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0066] Killackey E, Allot KA, Cotton SM, Chinnery GL, Sun P, Collins Z, et al. Recovery in first‐episode psychosis: first results from a large randomized controlled trial of IPS. Early Intervention in Psychiatry. 2012; Vol. 6, issue Suppl. 1:13.

[CD011867-bib-0067] Killackey E, Allott KA, Cotton S, Chinnery GL, Jackson H. Baseline to 18 months: main results from a randomized controlled trial of individual placement and support for young people with first‐episode psychosis. Early Intervention in Psychiatry. 2014; Vol. 8, issue Suppl. 1:152.

[CD011867-bib-0068] Latimer EA, LecomteT, Becker DR, Drake RE, Duclos I, Piat M, et al. Generalisability of the individual placement and support model of supported employment: results of a Canadian randomised controlled trial. British Journal of Psychiatry 2006;189:65‐73. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0069] Lecomte T, Corbière M, Lysaker PH. A group cognitive behavioral intervention for people registered in supported employment programs: CBT‐SE [Une intervention cognitive comportementale de groupe pour les personnes suivies dans le cadre d’un programme de soutien en emploi (TCC‐SE)]. L’Encéphale 2014;40:S81‐S90. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0070] Gold JM, Goldberg RW, McNary SW, Dixon LB, Lehman AF. Cognitive correlates of job tenure among patients with severe mental illness. American Journal of Psychiatry 2002;159:1359‐402. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0071] Lehman AF, Goldberg R, Dixon LB, McNary S, Postrado L, Hackman A, et al. Improving employment outcomes for persons with severe mental illnesses. Archives of General Psychiatry 2002;59(2):165‐72. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0072] McFarlane WR, Dushay RA, Stastny P, Deakins SM, Link B. A comparison of two levels of family‐aided assertive community treatment. Psychiatric Services 1996;47:744‐50. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0073] McFarlane WR, Dushay RA, Deakins SM, Stasny P, Lukens EP, Toran J, et al. Employment outcomes in family‐aided assertive community treatment. Journal of Orthopsychiatry 2000;70(2):203‐14. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0074] McGurk SR, Mueser KT, Pascaris A. Cognitive training and supported employment for persons with severe mental illness: one‐year results from a randomized controlled trial. Schizophrenia Bulletin 2005;31(4):898‐909. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0075] McGurk, SR, Mueser KT, Feldman K, Wolfe R, Pascaris A. Cognitive training for supported employment: 2‐3 year outcomes of a randomized controlled trial. American Journal of Psychiatry 2007;164:437‐41. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0076] McGurk SR, Mueser KT, DeRosa TJ, Wolfe R. Work, recovery, and comorbidity in schizophrenia: a randomized controlled trial of cognitive remediation. Schizophrenia Bulletin 2009;35(2):319‐35. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD011867-bib-0077] Michon H, Busschbach JT, Stant AD, Vugt MD, Weeghel J, Kroon H. Effectiveness of individual placement and support for people with severe mental illness in the Netherlands: a 30‐month randomized controlled trial. Psychiatric Rehabilitation Journal 2014;37(2):129‐36. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0078] Stant AD, Busschbach JT, Vugt M, Michon H. A rehabilitation intervention to help people with severe mental illness obtain and keep a paid job: the economic evaluation. Journal of Mental Health Policy and Economics 2013;6:S32. [Google Scholar]

[CD011867-bib-0079] Busschbach JT, Michon H, Vugt M, Stant AD, Aerts‐Roorda LC, Erp N. Effectiveness of individual placement and support in the Netherlands. Final report of a randomised controlled trial. Part 2: Findings after 30 months follow‐up [Effectiviteit van Individuele Plaatsing en Steun in Nederland. Eindverslag van een gerandomiseerde gecontroleerde effectstudie Deel 2: Bevindingen na 30 maanden follow‐up]. Rob Giel Onderzoekcentrum, Groningen & Trimbos‐instituut, Utrecht (Dutch research report) 2011.

[CD011867-bib-0080] Mueser KT, Becker DR, Wolfe R. Supported employment, job preferences, job tenure and satisfaction. Journal of Mental Health 2001;10(4):411‐17. [Google Scholar]

[CD011867-bib-0081] Mueser KT, Bond GR, Essock SM, Clark RE, Carpenter‐Song E, Drake RE, et al. The effects of supported employment in Latino consumers with severe mental illness. Psychiatric Rehabilitation Journal 2014;37(2):113‐22. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0082] Mueser KT, Clark RE, Haines M, Drake RE, McHugo GJ, Bond GR, et al. The Hartford study of supported employment for persons with severe mental illness. Journal of Consulting and Clinical Psychology 2004;72(3):479‐90. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0083] Mueser KT, Essock SM, Haines M, Wolfe R, Xie H. Posttraumatic stress disorder, supported employment, and outcomes in people with severe mental illness. CNS Spectrums 2004;9(12):913‐25. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0084] Bond GR, Drake RE, Luciano A. Employment and educational outcomes in early intervention programmes for early psychosis: a systematic review. Epidemiology and Psychiatric Sciences 2015;24(5):446‐57. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD011867-bib-0085] Nuechterlein KH, Subotnik KL, Turner LR, Ventura J, Becker DR, Drake RE. Individual placement and support for individuals with recent‐onset schizophrenia: Integrating supported education and supported employment. Psychiatric Rehabilitation Journal 2008;31(4):340‐9. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0086] Nuechterlein KH, Subotnik KL, Turner LR, Ventura J, Gitlin MJ, Gretchen‐Doorly D, et al. Individual placement and support after an initial episode of schizophrenia: impact on school or work recovery, hospitalization and utilization of disability support. Early Intervention in Psychiatry. 2012; Vol. 6, issue Suppl. 1:14.

[CD011867-bib-0087] O’Brien A, Price C, Burns T, Perkins C. Improving the vocational status of patients with long‐term mental illness: a randomised controlled trial of staff training. Community Mental Health Journal 2003;39(4):333‐47. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0088] Oshima I, Sono T, Bond GR, Nishio M, Ito J. A randomized controlled trial of individual placement and support in Japan. Psychiatric Rehabilitation Journal 2014;37(2):137‐43. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0089] Penk W, Drebing CE, Rosenheck RA, Krebs C, Ormer A, Mueller L. Veterans Health Administration transitional work experience vs. job placement in veterans with co‐morbid substance use and non‐psychotic psychiatric disorders. Psychatric Rehabilitation Journal 2010;33(4):297‐307. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0090] Gold PB, Macias C, Rodican CF. Does competitive work improve quality of life for adults with severe mental illness? Evidence from a randomized trial of supported employment. The Journal of Behavioral Health Services & Research 2016;43(2):155‐71. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD011867-bib-0091] Macias C, DeCarlo LT, Wang Q, Frey J, Barreira P. Work interest as a predictor of competitive employment: policy implications for psychiatric rehabilitation. Administration and Policy in Mental Health 2001;28(4):279‐97. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD011867-bib-0092] Macias C, Rodican CF, Hargreaves WA, Jones DR, Barreira PJ, Wang Q. Supported employment outcomes of a randomized controlled trial of ACT and Clubhouse models. Psychiatric Services 2006;57(10):1406‐15. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD011867-bib-0094] Schonebaum AD, Boyd JK. Work‐ordered day as a catalyst of competitive employment success. Psychiatric Rehabilitation Journal 2012;35(5):391. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0093] Schonebaum AD, Boyd JK, Dudek KJ. A comparison of competitive employment outcomes for the Clubhouse and PACT models. Psychiatric Services 2006;57(10):1416‐20. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0096] Tsang HWH. Rehab rounds: social skills training to help mentally ill persons find and keep a job. Psychiatric Services 2001;52(7):891‐4. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0095] Tsang HWH, Pearson V. Work‐related social skills training for people with schizophrenia in Hong Kong. Schizophrenia Bulletin 2001;27(1):139‐48. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0099] Tsang HWH. Supported employment versus traditional vocational rehabilitation for individuals with severe mental illness: a three‐year study. Hong Kong Medical Journal 2011;17(Suppl 2):S13‐7. [PubMed] [Google Scholar]

[CD011867-bib-0097] Tsang HWH, Chan A, Wong A, Liberman RP. Vocational outcomes of an integrated supported employment program for individuals with persistent and severe mental illness. Journal of Behavior Therapy and Experimental Psychiatry 2009;40(2):292‐305. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0098] Tsang HWH, Fung KM, Leung AY, Li SM, Cheung WM. Three year follow‐up study of an integrated supported employment for individuals with severe mental illness. Australian and New Zealand Journal of Psychiatry 2010;44(1):49‐58. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0100] Twamley EW, Narvaez JM, Becker DR, Bartels SJ, Jeste DV. Supported employment for middle‐aged and older people with schizophrenia. American Journal of Psychiatric Rehabilitation 2008;11:76‐89. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD011867-bib-0101] Twamley EW, Vella L, Burton CZ, Becker DR, Bell MD, Jeste DV. The efficacy of supported employment for middle‐aged and older people with schizophrenia. Schizophrenia Research 2012;135:100‐4. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD011867-bib-0102] Landolt K, Brantchen E, Nordt C, Bärtch B, Kawohl W, Rössler W. Associations of supported employment with cognitive functioning and employment outcomes. Psychiatric Services 2016;67(11):1257‐61. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0103] Viering S, Bärtsch B, Obermann C, Rüsch N, Rössler W, Kawohl W. The effectiveness of individual placement and support for people with mental illness new on social benefits: a study protocol. BMC Psychiatry 2013;13:195. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD011867-bib-0104] Viering S, Jäger M, Bärtsch B, Nordt C, Rössler W, Warnke I, et al. Supported employment for the reintegration of disability pensioners with mental illnesses: a randomized controlled trial. Frontiers in Public Health 2015;3(237):1‐7. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD011867-bib-0105] Waghorn G, Childs S, Hampton E, Gladman B, Greaves A, Bowman D. Enhancing community mental health services through formal partnerships with supported employment providers. American Journal of Psychiatric Rehabilitation 2012;15:157‐80. [Google Scholar]

[CD011867-bib-0106] Waghorn G, Dias S, Gladman B, Harris M, Saha S. A multi‐site randomised controlled trial of evidence‐based supported employment for adults with severe and persistent mental illness. Australian Occupational Therapy Journal 2014;61:424‐36. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0107] Walker R, Winick W, Frost ES, Lieberman JM. Social restoration of hospitalised psychiatric patients through a program of special employment in industry. Rehabilitation Literature 1969;30(10):297‐303. [PubMed] [Google Scholar]

[CD011867-bib-0108] Wong K, Chiu R, Tang B, Mak D, Liu J, Chiu SN. A randomized controlled trial of a supported employment program for persons with long‐term mental illness in Hong Kong. Psychiatric Services 2008;59(1):84‐90. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0109] Xiang YT, Weng YZ, Li WY, Gao L, Chen GL, Xie L, et al. Efficacy of the Community Re‐Entry Module for patients with schizophrenia in Beijing, China: outcome at 2‐year follow‐up. British Journal of Psychiatry 2007;190:49‐56. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0110] Audini B, Marks IM, Lawrence, RE, Connolly J, Watts V. Home‐based versus out‐patient/in‐patient care for people with serious mental illness. Phase II of a controlled study. The British Journal of Psychiatry 1994;165(2):204‐10. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0111] Knapp M, Marks I, Wolstenholme J, Beecham J, Astin, J, Audini B, et al. Home‐based versus hospital‐based care for serious mental illness. Controlled cost‐effectiveness study over four years. The British Journal of Psychiatry 1998;172(6):506‐12. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0112] Bateman A, Fonagy P. 8‐year follow‐up of patients treated for borderline personality disorder: mentalization‐based treatment versus treatment as usual. The American Journal of Psychiatry 2008;165(5):631‐8. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0113] Bateman A, Fonagy P. Effectiveness of partial hospitalization in the treatment of borderline personality disorder: a randomized controlled trial. American Journal of Psychiatry 1999;156:1563‐9. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0114] Bateman A, Fonagy P. Treatment of borderline personality disorder with psychoanalytically oriented partial hospitalization: an 18‐month follow‐up. American Journal of Psychiatry 2001;158:36‐42. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0115] Bayer W, Köster M, Salize HJ, Höhl W, Machleidt W, Wiedl KH, et al. Longer‐term effects of inpatient vocational and ergotherapeutic measures on the vocational integration of patients with schizophrenia [Längerfristige Auswirkungen stationärer arbeits‐ und ergotherapeutischer Maßnahmen auf die berufliche Integration schizophrener Patienten]. Psychiatrische Praxis 2008;35(4):170‐4. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0116] Becker D, Whitley R, Bailey EL, Drake RE. Long‐term employment trajectories among participants with severe mental illness in supported employment. Psychiatric Services 2007;58(7):922‐8. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0118] Bell MD, Lysaker PH. Clinical benefits of paid work activity in schizophrenia: 1‐year follow up. Schizophrenia Bulletin 1997;23(2):317‐28. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0117] Bell MD, Lysaker PH, Milstein RM. Clinical benefits of paid work activity in schizophrenia. Schizophrenia Bulletin 1996;22(1):51‐67. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0119] Bell MD, Milstein RM, Lysaker PH. Pay and participation in work activity: clinical benefits for clients with schizophrenia. Psychosocial Rehabilitation Journal 1993;17(2):173‐7. [Google Scholar]

[CD011867-bib-0120] Bell MD, Milstein RM, Lysaker PH. Pay as an incentive in work participation by patients with severe mental illness. Psychiatric Services 1993;44(7):684‐6. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0121] Bell M, Lysaker P, Bryson G. A behavioral intervention to improve work performance in schizophrenia: work behavior inventory feedback. Journal of Vocational Rehabilitation 2003;1:43‐50. [Google Scholar]

[CD011867-bib-0122] Bell MD, Bryson G, Greig T, Corcoran C, Wexler BE. Neurocognitive enhancement therapy with work therapy: effects on neuropsychological test performance. Archives of General Psychiatry 2001;8:763‐8. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0123] Bell MD, Bryson G, Wexler BE. Cognitive remediation of working memory deficits: durability of training effects in severely impaired and less severely impaired schizophrenia. Acta Psychiatrica Scandinavica 2003;2:101‐9. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0124] Bell MD, Bryson GJ, Greig TC, Fiszdon JM, Wexler BE. Neurocognitive enhancement therapy with work therapy: productivity outcomes at 6‐ and 12‐month follow‐ups. Journal of Rehabilitation Research and Development 2005;6:829‐38. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0125] Bell MD, Fiszdon J, Greig T, Wexler BE, Bryson G. Neurocognitive enhancement therapy with work therapy in schizophrenia: 6‐month follow‐up of neuropsychological performance. Journal of Rehabilitation Research and Development 2007;5:761‐70. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0126] Bell MD, Tsang HW, Greig TC, Bryson GJ. Neurocognition, social cognition, perceived social discomfort, and vocational outcomes in schizophrenia. Schizophrenia Bulletin 2009;35(4):738‐47. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD011867-bib-0127] Fiszdon JM, Bell MD. Cognitive remediation and work therapy in the outpatient treatment of patients with schizophrenia [Remédiation cognitive et thérapie occupationnelle dans le traitement ambulatoire du patient souffrant de schizophrénie]. Santé Mentale au Québec 2004;29(2):117‐42. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0128] Bell MD, Zito W, Greig T, Wexler BE. Neurocognitive enhancement therapy with vocational services: work outcomes at two‐year follow‐up. Schizophrenia Research 2008;1‐3:18‐29. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0129] Bell MD, Zito W, GreigT, Wexler BE. Neurocognitive enhancement therapy and competitive employment in schizophrenia: effects on clients with poor community functioning. American Journal of Psychiatric Rehabilitation 2008;11(2):109‐22. [Google Scholar]

[CD011867-bib-0130] Greig TC, Zito W, Wexler BE, Fiszdon J, Bell MD. Improved cognitive function in schizophrenia after one year of cognitive training and vocational services. Schizophrenia Research 2007;96(1‐3):156‐61. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD011867-bib-0131] Bell MD, Choi KH, Dyer C, Wexler BE. Benefits of cognitive remediation and supported employment for schizophrenia patients with poor community functioning. Psychiatric Services 2014;65(4):469‐75. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0132] Bell MD, Corbera S, Wexler BE. Cognitive remediation and competitive employment: differential benefits for schizophrenia patients with poor community function. Schizophrenia Research 2012;136:S78. [Google Scholar]

[CD011867-bib-0133] Bell MD, Corbera S, Wexler BE. Cognitive remediation and supported employment: moderators and mediators of vocational outcomes. Schizophrenia Bulletin 2013;39:S281‐S282. [Google Scholar]

[CD011867-bib-0134] Bertelsen M, Jeppesen P, Petersen L, Thorup A, Øhlenschlaeger J, Quach P, Christensen TØ, Krarup G, et al. First episode of psychosis intensive early intervention programme versus standard treatment‐‐secondary publication. Ugeskrift for Laeger 2009;171(41):2992‐5. [PubMed] [Google Scholar]

[CD011867-bib-0135] Bertelsen M, Jeppesen P, Petersen L, Thorup A, Øhlenschlaeger J, Quach P, et al. Five‐year follow‐up of a randomized multicenter trial of intensive early intervention vs standard treatment for patients with a first episode of psychotic illness: the OPUS trial. Archives of General Psychiatry 2008;65(7):762‐71. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0136] Bonebrake K, Bertelsen M, Thoru A, Pi Jeppesen P, Albert N, Krarup G, et al. Use of a latent variable framework to examine the relationship between symptom presentation at baseline and recovery at 5 years. Early Intervention in Psychiatry 2010;4:55. [Google Scholar]

[CD011867-bib-0137] Hastrup LH, Kronborg C, Bertelsen M, Jeppesen P, Jorgensen P, Petersen L, et al. Cost‐effectiveness of early intervention in first‐episode psychosis: economic evaluation of a randomised controlled trial (the OPUS study). British Journal of Psychiatry 2013;1:35‐41. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0138] Jørgensen P, Nordentoft M, Abel MB, Gouliaev G, Jeppesen P, Kassow P. Early detection and assertive community treatment of young psychotics: the Opus study rationale and design of the trial. Social Psychiatry and Psychiatric Epidemiology 2000;35(7):283‐7. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0139] Nordentoft M, Bertelsen M, Albert N, Jeppesen P, Petersen L, Thorup A, et al. The OPUS trial: a randomized multicentre single‐blinded trial of specialized assertive early intervention (OPUS Treatment) versus standard treatment for patients with a first episode of psychotic illness ‐ five‐year follow‐up. Early Intervention in Psychiatry 2010;4:24. [Google Scholar]

[CD011867-bib-0140] Nordentoft M, Melau M, Iversen T, Petersen L, Jeppesen P, Thorup A, et al. From research to practice: how OPUS treatment was accepted and implemented throughout Denmark. Early Intervention in Psychiatry 2015;9(2):156‐62. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0141] Thorup A, Albert N, Bertelsen M, Petersen L, Jeppesen P, Quack P, et al. Gender differences in first‐episode psychosis at 5‐year follow‐up – two different courses of disease? Results from the OPUS study at 5‐year follow‐up. European Psychiatry 2015;29(1):44‐51. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0142] Bond GR, Drake RE, Campbell K. Effectiveness of individual placement and support supported employment for young adults. Early Intervention in Psychiatry 2016;10(4):300‐7. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD011867-bib-0143] Burke‐Miller JK, Cook JA, Grey DD, Razzano LA, Blyler CR, Leff HS, et al. Demographic characteristics and employment among people with severe mental illness in a multisite study. Community Mental Health Journal 2006;42(2):143‐59. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0144] Cook JA, Blyler CR, Burke‐Miller JK, McFarlane WR, Leff HS, Mueser KT, et al. Effectiveness of supported employment for individuals with schizophrenia: results of a multi‐site, randomized trial. Clinical Schizophrenia & Related Psychoses 2008;1:37‐46. [Google Scholar]

[CD011867-bib-0145] Cook JA, Blyler CR, Leff HS, McFarlane WR, Goldberg RW, Gold PB, et al. The employment intervention demonstration program: major findings and policy implications. Psychiatric Rehabilitation Journal 2008;31(4):291‐5. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0146] Cook JA, Carey MA, Razzano LA, Burke J, Blyler CR. The pioneer: the employment intervention demonstration program. New Directions for Evaluation 2002;94:31‐44. [Google Scholar]

[CD011867-bib-0147] Cook JA, Leff HS, Blyler CR, Gold PB, Goldberg RW, Mueser KT, et al. Results of a multi site randomized trial of supported employment interventions for individuals with severe mental illness. Archives of General Psychiatry 2005;5:505‐12. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0148] Cook JA, Lehman AF, Drake R, McFarlane WR, Gold PB, Leff HS, et al. Integration of psychiatric and vocational services: a multisite randomized controlled trial of supported employment. American Journal of Psychiatry 2005;10:1948‐56. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0149] Cook JA, Mulkern V, Grey DD, Burke‐Miller J, Blyler CR, Razzano LA, et al. Effects of local unemployment rate on vocational outcomes in a randomized trial of supported employment for individuals with psychiatric disabilities. Journal of Vocational Rehabilitation 2006;25(2):71‐84. [Google Scholar]

[CD011867-bib-0150] Cook JA, Razzano LA, Burke‐Miller JK, Blyler CR, Leff HS, Mueser KT, et al. Effects of co‐occurring disorders on employment outcomes in a multisite randomized study of supported employment for people with severe mental illness. Journal of Rehabilitation Research and Development 2007;6:837‐49. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0151] Razzano LA Cook JA, Burke‐Miller JK, Mueser KT, Pickett‐Schenk SA, Grey DD, et al. Clinical factors associated with employment among people with severe mental illness: findings from the employment intervention demonstration program. Journal of Nervous and Mental Disease 2005;193(11):705‐13. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0152] Cook S, Chambers E, Coleman JH. Occupational therapy for people with psychotic conditions in community settings: a pilot randomized controlled trial. Clinical Rehabilitation 2009;23(1):40‐52. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0153] Davis LL, Leon AC, Toscano R, Drebing CE, Ward LC, Parker PE, et al. A randomized controlled trial of supported employment among veterans with posttraumatic stress disorder. Psychiatric Services 2012;63(5):464‐70. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0154] Davis LL, Pilkinton P, Poddar S, Blansett C, Toscano R, Parker PE. Impact of social challenges on gaining employment for veterans with posttraumatic stress disorder: an exploratory moderator analysis. Psychiatric Rehabilitation Journal 2014;37(2):107‐9. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0155] Davis LW, Lysaker PH, Eicher AC. Effects of a mindfulness intervention on work outcomes for adults with schizophrenia. Schizophrenia Bulletin 2011;37:262‐3. [Google Scholar]

[CD011867-bib-0156] Davis LW, Lysaker PH, Kristeller JL, Salyers MP, Kovach AC, Woller S. Effect of mindfulness on vocational rehabilitation outcomes in stable phase schizophrenia. Psychological Services 2015;12(3):303‐12. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0157] Fowler D, Hodgekins J, Painter M, Reilly T, Crane C, Macmillan I, et al. Cognitive behaviour therapy for improving social recovery in psychosis: a report from the ISREP MRC Trial Platform study (Improving Social Recovery in Early Psychosis). Psychological Medicine 2009;39:1627‐36. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0158] Granholm E, Holden J, Link PC, McQuaid JR. Randomized clinical trial of cognitive behavioral social skills training for schizophrenia: improvement in functioning and experiential negative symptoms. Journal of Consulting and Clinical Psychology 2014;82(6):1173‐85. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD011867-bib-0159] Griffiths RD. Rehabilitation of chronic psychotic patients. An assessment of their psychological handicap, an evaluation of the effectiveness of rehabilitation, and observations of the factors which predict outcome. Psycholical Medicine 1974;4(3):316‐25. [PubMed] [Google Scholar]

[CD011867-bib-0160] Hamilton SH, Edgell ET, Revicki DA, Breier A. Functional outcomes in schizophrenia: a comparison of olanzapine and haloperidol in a European sample. International Clinical Psychopharmacology 2000;15(5):245‐55. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0161] Haslett WR, McHugo GJ, Bond GR, Drake RE. Use of software for tablet computers to promote engagement with supported employment: results from an RCT. Psychiatric Services 2014;65(7):954‐6. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0162] Hirschfeld RM, Dunner DL, Keitner G, Klein DN, Koran LM, Kornstein SG, et al. Does psychosocial functioning improve independent of depressive symptoms? A comparison of nefazodone, psychotherapy, and their combination. Biological Psychiatry 2002;51(2):123‐33. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0163] Hogarty GE, Flesher S, Ulrich R, Carter M, Greenwald D, Pogue‐Geile M, et al. Archives of General Psychiatry. Cognitive enhancement therapy for schizophrenia: effects of a 2‐year randomized trial on cognition and behavior 2004;61(9):866‐76. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0164] Jager M, Paras S, Nordt C, Warnke I, Bartsch B, Rossler W, Kawohl W. How sustainable is supported employment? A follow‐up investigation. Neuropsychiatrie 2013;27(4):196‐201. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0165] Kidd SA, Kaur J, Virdee G, George TP, McKenzie K, Herman Y. Cognitive remediation for individuals with psychosis in a supported education setting: a randomized controlled trial. Schizophrenia Research 2014;157(1):90‐8. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0166] Kline MN, Hoisington V. Placing the psychiatrically disabled: a look at work values. Rehabilitation Counseling Bulletin 1981;24(5):366‐9. [Google Scholar]

[CD011867-bib-0167] Kopelowicz A. Adapting social skills training for Latinos with schizophrenia. International Review of Psychiatry 1998;10:47‐50. [Google Scholar]

[CD011867-bib-0168] Kuldau JM, Dirks SJ. Controlled evaluation of a hospital‐originated community transitional system. Archives of General Psychiatry 1977;34(11):1331‐40. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0169] Kurtz MM, Mueser KT, Wexler B. Cognitive remediation and social skills training for schizophrenia. Schizophrenia Bulletin 2013;39:S339‐S340. [Google Scholar]

[CD011867-bib-0170] Kurtz MM, Mueser KT, Thime WR, Corbera S, Wexler BE. Social skills training and computer‐assisted cognitive remediation in schizophrenia. Schizophrenia Research 2015;162(1‐3):35‐41. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD011867-bib-0171] Liberman RP, Wallace CJ, Blackwell G, Kopelowicz A, Vaccaro JV, Mintz J. Skills training versus psychosocial occupational therapy for persons with persistent schizophrenia. American Journal of Psychiatry 1998;155:1087‐91. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0172] Lindenmayer JP, McGurk SR, Mueser KT, Khan A, Wance D, Hoffman L, et al. Randomized controlled trial of cognitive remediation among inpatients with persistent mental illness. Psychiatric Services 2008;59(3):241‐7. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0173] Lucca AM, Henry AD, Banks S, Simon L, Page S. Evaluation of an Individual Placement and Support model (IPS) program. Psychiatric Rehabilitation Journal 2004;27(3):251‐7. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0174] Luo K, Yu D. Enterprise‐based sheltered workshops in Nanjing. A new model for the community rehabilitation of mentally ill workers. British Journal of Psychiatry 1994;24:89‐95. [PubMed] [Google Scholar]

[CD011867-bib-0175] Davis LW, Ringer JM, Strasburger AM, Lysaker PH. Participant evaluation of a CBT program for enhancing work function in schizophrenia. Psychiatric Rehabilitation Journal 2008;32(1):55‐8. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0176] Lysaker PH, Bond G, Davis LW, Bryson GJ, Bell MD. Enhanced cognitive‐behavioral therapy for vocational rehabilitation in schizophrenia: effects on hope and work. Journal of Rehabilitation Research and Development 2005;42(5):673‐82. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0177] Lysaker PH, Davis LW, Beattie N. Effects of cognitive behavioral therapy and vocational rehabilitation on metacognition and coping in schizophrenia. Journal of Contemporary Psychotherapy 2006;1:25‐30. [Google Scholar]

[CD011867-bib-0178] Lysaker PH, Davis LW, Beattie N. Effects of cognitive behavioral therapy and vocational rehabilitation on metacognition and coping in schizophrenia: Erratum. Journal of Contemporary Psychotherapy 2007;37(2):115. [Google Scholar]

[CD011867-bib-0179] Kukla M, Davis LW, Lysaker PH. Cognitive behavioral therapy and work outcomes: correlates of treatment engagement and full and partial success in schizophrenia. Behavioural and Cognitive Psychotherapy 2014;42(5):577‐92. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0180] Lysaker PH, Davis LW, Bryson GJ, Bell MD. Effects of cognitive behavioral therapy on work outcomes in vocational rehabilitation for participants with schizophrenia spectrum disorders. Schizophrenia Research 2009;107(2‐3):186‐91. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0181] Lysaker PH, Dimaggio G, Carcione A, Procacci M, Buck KD, Davis LW, et al. Metacognition and schizophrenia: the capacity for self‐reflectivity as a predictor for prospective assessments of work performance over six months. Schizophrenia Research 2010;122(1‐3):124‐30. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0182] Man DW, Law KM, Chung RC. Cognitive training for Hong Kong Chinese with schizophrenia in vocational rehabilitation. Hong Kong Medical Journal 2012;18(Suppl 6):18‐22. [PubMed] [Google Scholar]

[CD011867-bib-0183] Carrión RE, Cornblatt BA, Burton CZ, Tso IF, Auther AM, Adelsheim S, et al. Personalized prediction of psychosis: external validation of the NAPLS‐2 Psychosis Risk Calculator with the EDIPPP Project. American Journal of Psychiatry 2016;173(10):989‐96. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD011867-bib-0184] Lynch S, McFarlane WR, Joly B, Adelsheim S, Auther A, Cornblatt BA, et al. Early detection, intervention and prevention of psychosis program: community outreach and early identification at six U.S. sites. Psychiatric Services (Washington, D.C.) 67;5:510‐6. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0185] McFarlane W. Early detection and intervention for the prevention of psychosis (EDIPPP): a national multisite effectiveness trial of indicated prevention in the USA. Early Intervention in Psychiatry 2012;6:2. [Google Scholar]

[CD011867-bib-0186] McFarlane WL, Cook W, Downing D, Ruff A, Lynch S, Adelsheim S, et al. Early detection, intervention, and prevention of psychosis program: rationale, design, and sample description. Adolescent Psychiatry 2012;2(2):112‐24. [Google Scholar]

[CD011867-bib-0187] McFarlane WR, Levin B, Travis L, Lucas FL, Lynch S, Verdi M, et al. Clinical and functional outcomes after 2 years in the early detection and intervention for the prevention of psychosis multisite effectiveness trial. Schizophrenia Bulletin 2015;41(1):30‐43. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD011867-bib-0188] Tso IF, Taylor SF, Grove TB, Niendam T, Adelsheim S, Auther A, et al. Factor analysis of the Scale of Prodromal Symptoms: data from the Early Detection and Intervention for the Prevention of Psychosis Program. Early Intervention Psychiatry 2017;11(1):14‐22. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD011867-bib-0189] McGrew JH, Johannesen JK, Griss ME, Born DL, Katuin C. Performance‐based funding of supported employment: a multi‐site controlled trial. Journal of Vocational Rehabilitation 2005;2:81‐99. [Google Scholar]

[CD011867-bib-0190] McGrew JH, Johannesen JK, Griss ME, Born DL, Katuin CH. Performance‐based funding of supported employment for persons with severe mental illness: vocational rehabilitation and employment staff perspectives. Journal of Behavioral Health Services and Research 2007;34(1):1‐16. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0192] McGurk SR, Mueser KT. Cognitive and clinical predictors of work outcomes in clients with schizophrenia receiving supported employment services: 4‐year follow‐up. Administration and Policy in Mental Health and Mental Health Services Research 2006;33(5):598‐606. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0191] McGurk SR, Mueser KT, Harvey PD, LaPuglia R, Marder J. Cognitive and symptom predictors of work outcomes for clients with schizophrenia in supported employment. Psychiatric Services 2003;54:1129‐35. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0193] Mueser KT, Aalto S, Becker DR, Ogden JS, Wolfe JS, Schiavo D, Wallace CJ, Xie H. The effectiveness of skills training for improving outcomes in supported employment. Psychiatric Services 2005;56(10):1254‐60. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0194] Mueser KT, Campbell K, Drake RE. The effectiveness of supported employment in people with dual disorders. Journal of Dual Diagnosis 2011;7(1‐2):90‐102. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD011867-bib-0195] Okpaku SO, Anderson KH, Sibulkin AE, Butler JS, Bickman L. The effectiveness of a multidisciplinary case management intervention on the employment of SSDI applicants and beneficiaries. Psychiatric Rehabilitation Journal 1997;20(3):34‐41. [Google Scholar]

[CD011867-bib-0196] Resnick SG, Rosenheck RA, Canive JM, Souza C, Stroup TS, McEvoy J, et al. Employment outcomes in a randomized trial of second‐generation antipsychotics and perphenazine in the treatment of individuals with schizophrenia. Journal of Behavioral Health Services & Research 2008;2:215‐25. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0197] Rinaldi M, Perkins R, McNeil K, Hickman N, Singh SP. The individual placement and support approach to vocational rehabilitation for young people with first episode psychosis in the UK. Journal of Mental Health 2010;19(6):483‐91. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0198] Roder V, Brenner HD, Müller D, Lächler M, Zorn P, Reisch T, et al. Development of specific social skills training programmes for schizophrenia patients: Results of a multicentre study. Acta Psychiatrica Scandinavica 2002;105(5):363‐71. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0199] Rogers ES, Anthony WA, Lyass A, Penk WE. A randomized clinical trial of vocational rehabilitation for people with psychiatric disabilities. Counseling Bulletin 2006;49(3):143‐56. [Google Scholar]

[CD011867-bib-0200] Rosen MI, Ablondi K, Black AC, Mueller L, Serowik KL, Martino S, et al. Work outcomes after benefit counselling among veterans applying for service connection for a psychiatric condition. Psychiatric Services 2014;65(12):1426‐32. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD011867-bib-0201] Rus‐Calafell M, Gutiérrez‐Maldonado J, Ortega‐Bravo M, Ribas‐Sabaté J, Caqueo‐Urízar A. A brief cognitive‐behavioural social skills training for stabilised outpatients with schizophrenia: a preliminary study. Schizophrenia Research 2013;143(2‐3):372‐36. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0202] Sato S, Iwata K, Furukawa SI, Matsuda Y, Hatsuse N, Ikebuchi E. The effects of the combination of cognitive training and supported employment on improving clinical and working outcomes for people with schizophrenia in Japan. Clinical Practice and Epidemiology in Mental Health 2014;10:18‐27. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD011867-bib-0203] Shi L, Namjoshi MA, Zhang F, Gandhi G, Edgell ET, Tohen M, et al. Open‐label olanzapine treatment in bipolar I disorder: clinical and work functional outcomes. International Clinical Psychopharmacology 2002;17(5):227‐37. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0204] Sungur M, Soygur H, Guner P, Ustun B, Cetin I, Falloon IR. Identifying an optimal treatment for schizophrenia: a 2‐year randomized controlled trial comparing integrated care to a high‐quality routine treatment. International Journal of Psychiatry in Clinical Practice 2011;15(2):118‐27. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0205] Swildens W, Busschbach JT, Michon H, Kroon H, Koeter MW, Wiersma D, et al. Effectively working on rehabilitation goals: 24‐month outcome of a randomized controlled trial of the Boston psychiatric rehabilitation approach. The Canadian Journal of Psychiatry 2011;56(12):751‐60. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0206] Thunnissen M, Duivenvoorden H, Busschbach J, Hakkaart‐van Roijen L, Tilburg W, Verheul R, et al. A randomized clinical trial on the effectiveness of a reintegration training program versus booster sessions after short‐term inpatient psychotherapy. Journal of Personality Disorders 2008;22(5):483‐95. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0207] Torrent C, Bonnin CM, Martinez‐Aran A, Valle J, Amann BL, Gonzalez‐Pinto A, et al. Efficacy of functional remediation in bipolar disorder: a multicenter randomized controlled study. American Journal of Psychiatry 2013;170(8):852‐9. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0208] Trapp W, Landgrebe M, Hoesl K, Lautenbacher S, Gallhofer B, Günther W, et al. Cognitive remediation improves cognition and good cognitive performance increases time to relapse‐‐results of a 5 year catamnestic study in schizophrenia patients. BMC Psychiatry 2013;13:184. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD011867-bib-0209] Tsang MM, Man DW. A virtual reality‐based vocational training system (VRVTS) for people with schizophrenia in vocational rehabilitation. Schizophrenia Research 2013;144(1‐3):51‐62. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0210] Twamley EW, Padin DS, Bayne KS, Narvaez JM, Williams RE, Jeste DV. Work rehabilitation for middle‐aged and older people with schizophrenia: a comparison of three approaches. Journal of Nervous and Mental Disease 2005;193(9):596‐601. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0211] Twamley EW, Savla GN, Zurhellen CH, Heaton RK, Jeste DV. Development and pilot‐testing of a novel compensatory cognitive training intervention for people with psychosis. American Journal of Psychiatric Rehabilitation 2008;11:144‐63. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD011867-bib-0212] Twamley EW, Vella L, Burton CZ, Heaton RK, Jeste DV. Compensatory cognitive training for psychosis: effects in a randomized controlled trial. The Journal of Clinical Psychiatry 2012;73(9):1212‐19. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD011867-bib-0213] Vauth R, Corrigan, PW, Clauss M, Dietl M, Dreher‐Rudolph M, Stieglitz RD, et al. Cognitive strategies versus self‐management skills as adjunct to vocational rehabilitation. Schizophrenia Bulletin 2005;31(1):55‐66. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0214] Wolkon GH, Karmen M, Tanaka MHT. Evaluation of a social rehabilitation program for recently released psychiatric patients. Community Mental Health Journal 1971;7(4):312‐22. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0215] Xiang Y, Weng Y, Li W, Gao L, Chen G, Xie L, et al. Training patients with schizophrenia with the community re‐entry module. Social Psychiatry and Psychiatric Epidemiology 2006;41(6):464‐9. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0216] Bejerholm U, Larsson ME, Johanson S. Supported employment adapted for people with affective disorders—a randomized controlled trial. Journal of Affective Disorders 2017;207:212‐20. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0217] Glynn SM, Marder SR, Noordsy DL, O'Keefe C, Becker DR, Drake RE, et al. An RCT evaluating the effects of skills training and medication type on work outcomes among patients with schizophrenia. Psychiatric Services 2017;68(3):271‐7. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0218] Kane JM, Robinson DG, Schooler NR, Mueser KT, Penn DL, Rosenheck RA, et al. Comprehensive versus usual community care for first‐episode psychosis: 2‐year outcomes from the NIMH RAISE early treatment program. American Journal of Psychiatry 2015;173(4):362‐72. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD011867-bib-0219] Kane JM, Schooler NR, Marcy P, Correll CU, Brunette MF, Mueser KT, et al. The RAISE early treatment program for first‐episode psychosis: background, rationale, and study design. The Journal of Clinical Psychiatry 2015;76(3):240‐6. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD011867-bib-0220] Rosenheck R, Leslie D, Sint K, Lin H, Robinson DG, Schooler NR, et al. Cost‐effectiveness of comprehensive, integrated care for first episode psychosis in the NIMH RAISE early treatment program. Schizophrenia Bulletin 2016;42(4):896‐906. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD011867-bib-0221] Rosenheck R, Mueser KT, Sint K, Lin H, Lynde DW, Glynn SM, et al. Supported employment and education in comprehensive, integrated care for first episode psychosis: effects on work, school, and disability income. Schizophrenia Research 2017;182:1201‐128. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0222] McGurk SR, Mueser KT, Xie H, Welsh J, Kaiser S, Drake RE, et al. Cognitive enhancement treatment for people with mental illness who do not respond to supported employment: a randomized controlled trial. The American Journal of Psychiatry 2015;72(9):852‐61. [DOI: 10.1176/appi.ajp.2015.14030374] [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0223] McGurk SR, Mueser KT, Xie H, Feldman K, Shaya Y, Klein L, et al. Cognitive remediation for vocational rehabilitation nonresponders. Schizophrenia Research 2016;175(1‐3):48‐56. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0224] Schneider J, Akhtar A, Boycott N, Guo B, Latimer E, Cao Z, et al. Individual placement and support versus individual placement and support enhanced with work‐focused cognitive behaviour therapy: feasibility study for a randomised controlled trial. British Journal of Occupational Therapy 2016;79(5):257‐69. [Google Scholar]

[CD011867-bib-0225] Bell MD. Cognitive training to enhance work program outcomes: preliminary findings. Schizophrenia Bulletin 2015;41:S302. [Google Scholar]

[CD011867-bib-0226] Bitter NA, Roeg DP, Nieuwenhuizen C, Weeghel J. Effectiveness of the Comprehensive Approach to Rehabilitation (CARe) methodology: design of a cluster randomized controlled trial. BMC Psychiatry 2015;15(1):165. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD011867-bib-0227] Christensen TN, Nielsen IG, Stenager E, Morthorst BR, Linschou J, Nordentoft M, et al. Individual placement and support supplemented with cognitive remediation and work‐related social skills training in Denmark: study protocol for a randomized controlled trial. Trials 2015;16:280. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD011867-bib-0228] Granholm E, Holden JL, Sommerfeld D, Rufener C, Perivoliotis D, Mueser K, et al. Enhancing assertive community treatment with cognitive behavioral social skills training for schizophrenia: study protocol for a randomized controlled trial. Trials 16;1:438. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD011867-bib-0229] Harris AW, Kosic T, Walker C, Gye W, Redoblado‐Hodge A. Internet based cognitive remediation can assist people with severe mental illness to gain and retain employment‐the Cogrem study. Schizophrenia Bulletin 2015;41:S314. [Google Scholar]

[CD011867-bib-0230] Melau M, Jeppesen P, Thorup A, Bertelsen M, Petersen L, Gluud C, et al. The effect of five years versus two years of specialised assertive intervention for first episode psychosis ‐ OPUS II: study protocol for a randomized controlled trial. Trials 2011;12(1):1. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD011867-bib-0231] Nordt C, Brantschen E, Kawohl W, Bärtsch B, Haker H, Rüsch N, et al. ‘Placement budgets’ for supported employment–improving competitive employment for people with mental illness: study protocol of a multicentre randomized controlled trial. BMC Psychiatry 2012;1:165. [DOI: 10.1186/1471-244X-12-165] [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD011867-bib-0232] Sveinsdottir V, Løvvik C, Fyhn T, Monstad K, Ludvigsen, K, Øverland S, et al. Protocol for the effect evaluation of Individual Placement and Support (IPS): a randomized controlled multicenter trial of IPS versus treatment as usual for patients with moderate to severe mental illness in Norway. BMC Psychiatry 2014;1:307. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD011867-bib-0233] 34 C.F.R. § 363.3(b) 2015. What definitions apply?. retrieved data March 12, 2015 from www.ecfr.gov.

[CD011867-bib-0234] Almerie MQ, Okbar Al Marhi M, Jawoosh M, Alsabbagh M, Matar HE, Maayan N, et al. Social skills programmes for schizophrenia. Cochrane Database of Systematic Reviews 2015, Issue 6. [DOI: 10.1002/14651858.CD009006.pub2] [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD011867-bib-0235] Arbesman M, Logsdon DW. Occupational therapy interventions for employment and education for adults with serious mental illness: a systematic review. American Journal of Occupational Therapy 2011;65(3):238‐46. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0236] Beard JH, Propst RN, Malamud TJ. The Fountain House model of psychiatric rehabilitation. Psychosocial Rehabilitation Journal 1982;5(1):47‐53. [Google Scholar]

[CD011867-bib-0237] Becker DR, Drake RE. A working life: the individual placement and support (IPS) program. Concord NH: New Hampshire‐Dartmouth Psychiatric Research Center, 1993. [Google Scholar]

[CD011867-bib-0238] Becker DR, Drake RE, Bond GR, Xie H, Dain BJ, Harrison K. Job terminations among persons with severe mental illness participating in supported employment. Community Mental Health Journal 1998;34(1):71‐82. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0239] Becker MA, Diamond R. Wisconsin Quality of Life Index (W‐QLI): a multidimensional model for measuring quality of life. Journal of Clinical Psychiatry 1999;60(Suppl 3):29‐31. [PubMed] [Google Scholar]

[CD011867-bib-0240] Bell MD, Zito W, Greig T, Wexler BE. Neurocognitive enhancement therapy with vocational services: work outcomes at two‐year follow‐up. Schizophrenia Research 2008;105(1):18‐29. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0241] Bellack AS, Mueser KT. Psychosocial treatment for schizophrenia. Schizophrenia Bulletin 1993;19(2):317‐36. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0242] Berwick DM, Murphy JM, Goldman PA, Ware JE Jr, Barsky AJ, Weinstein MC. Performance of a five‐item mental health screening test. Medical Care 1991;29(2):169‐76. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0243] Bilby R. A response to the criticisms of transitional employment. Psychosocial Rehabilitation Journal 1992;16(2):69‐82. [Google Scholar]

[CD011867-bib-0244] Boardman J. Work, employment and psychiatric disabilities. Advances in Psychiatric Treatment 2003;9:327‐34. [Google Scholar]

[CD011867-bib-0245] Bond GR, Becker DR, Drake RE, Vogler KM. A fidelity scale for the individual placement and support model of supported employment. Rehabilitation Counseling Bulletin 1997;40(4):265‐84. [Google Scholar]

[CD011867-bib-0246] Bond GR, Drake RE, Becker DR, Mueser KT. Effectiveness of psychiatric rehabilitation approaches for employment of people with severe mental illness. Journal of Disability Policy Studies 1999;10(1):18‐52. [Google Scholar]

[CD011867-bib-0247] Bond GR, Resnick SG, Drake RE, Xie H, McHugo GJ, Bebout RR. Does competitive employment improve nonvocational outcomes for people with severe mental illness?. Journal of Consulting and Clinical Psychology 2001;69(3):489‐501. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0248] Bond GR, Drake RE, Mueser K, Latimer E. Assertive community treatment for people with severe mental illness. Disease Management and Health Outcomes 2001;9(3):141‐59. [Google Scholar]

[CD011867-bib-0249] Bond GR, Campbell K, Ecans LJ, Gervey R, Pascaris A, Tice S, et al. A scale to measure quality of supported employment for persons with severe mental illness. Journal of Vocational Rehabilitation 2002;17(4):239‐50. [Google Scholar]

[CD011867-bib-0250] Bond GR, Salyers MP. Prediction of outcome from the Dartmouth assertive community treatment fidelity scale. CNS Spectrums 2004;9(12):937‐42. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0251] Bond G, Drake RE. Making the case for IPS supported employment. Administration and Policy in Mental Health and Mental Health Services Research 2012;41(1):69‐73. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0252] Bond GR, Petersen AE, Becker DR, Drake RE. Validation of the revised individual placement and support fidelity scale (IPS‐25). Psychiatric Services 2012;63:758‐63. [the fidelity scale and manual are available at: www.dartmouth.edu/˜ips/page19/page19.html] [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0253] Bond GR, Drake RE, Luciano A. Employment and educational outcomes in early intervention programmes for early psychosis: a systematic review. Epidemiology and Psychiatric Sciences 2015;24(5):446‐57. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD011867-bib-0254] Bouvet C, Battin C, Le‐Roy‐Hatala C. The Clubhouse model for people with severe mental illnesses: literature review and French experiment [Le modèle Clubhouse pour les personnes souffrant de troubles psychiques: revue de littérature et expérience francaise]. L'Encéphale 2014;41(6):477‐86. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0255] Boycott N, Schneider J, McMurran M. Additional interventions to enhance the effectiveness of individual placement and support: a rapid evidence assessment. Rehabilitation Research and Practice 2012. [DOI: 10.1155/2012/382420] [DOI] [PMC free article] [PubMed]

[CD011867-bib-0256] Burns T, Catty J, White S, Becker T, Koletsi M, Fioritti A, et al. The impact of supported employment and working on clinical and social functioning: results of an international study of individual placement and support. Schizophrenia Bulletin 2009;35(5):949‐58. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD011867-bib-0257] Carlier BE, Schuring M, Lötters FJ, Bakker B, Borgers N, Burdorf A. The influence of re‐employment on quality of life and self‐rated health, a longitudinal study among unemployed persons in the Netherlands. BMC Public Health 2013;13(1):503. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD011867-bib-0258] Chaimani A, Higgins JP, Mavridis D, Spyridonos P, Salanti G. Graphical tools for network meta‐analysis in STATA. PloS one 2013;8(10):e76654. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD011867-bib-0259] Chan JY, Hirai HW, Tsoi KK. Can computer‐assisted cognitive remediation improve employment and productivity outcomes of patients with severe mental illness? A meta‐analysis of prospective controlled trials. Journal of Psychiatric Research 2015;68:293‐300. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0260] Corbière M, Lecomte T. Vocational services offered to people with severe mental illness. Journal of Mental Health 2009;18(1):38‐50. [Google Scholar]

[CD011867-bib-0261] Corbière M, Negrini A, Dewa C. Mental health problems and mental disorders ‐ linked determinants to work participation and work functioning. In: Loisel P, Anema JR editor(s). Handbook of Work Disability: Prevention and Management. New York: Springer Science+Business Media, 2013:267‐88. [Google Scholar]

[CD011867-bib-0262] Corbière M, Brouwers E, Lanctôt N, Weeghel J. Employment specialist competencies for supported employment programs. Journal of Occupational Rehabilitation 2014;24(3):484‐97. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0263] Corbière M, Lecomte T, Reinharz D, Kirsch B, Goering P, Menear M, et al. Predictors of acquisition of competitive employment for people enrolled in supported employment programs. Journal of Nervous and Mental Disease In press. [DOI] [PubMed]

[CD011867-bib-0264] Corrigan PW. Place‐then‐train: an alternative service paradigm for persons with psychiatric disabilities. Science and Practice 2001;18:334‐49. [Google Scholar]

[CD011867-bib-0265] Crowther R, Marshall M, Bond G, Huxley P. Vocational rehabilitation for people with severe mental illness. Cochrane Database of Systematic Reviews 2001, Issue 2. [DOI: 10.1002/14651858.CD003080] [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD011867-bib-0266] Delespaul PH. Consensus regarding the definition of persons with severe mental illness and the number of such persons in the Netherlands. Tijdschrift voor Psychiatrie 2013;55(6):427‐38. [PubMed] [Google Scholar]

[CD011867-bib-0267] Dieterich M, Irving CB, Park B, Marshall M. Intensive case management for severe mental illness. Cochrane Database of Systematic Reviews 2010, Issue 10. [DOI: 10.1002/14651858.CD007906.pub2] [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD011867-bib-0268] Dilk MN, Bond GR. Meta‐analytic evaluation of skills training research for individuals with severe mental illness. Journal of Consulting and Clinical Psychology 1996;64(6):1337–46. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0269] Drake RE, Becker DR, Clark RE, Mueser KT. Research on the individual placement and support model of supported employment. Psychiatric Quarterly 1999;70(4):289‐301. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0270] Drake RE, Bond GR, Becker DR. Individual placement and support: an evidence‐based approach to supported employment. Oxford University Press, 2012. [Google Scholar]

[CD011867-bib-0271] Gilbert E, Marwaha S, Milton A, Johnson S, Morant N, Parsons N, et al. Social firms as a means of vocational recovery for people with mental illness: a UK survey. BMC Health Servey Research 2013;13:270. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD011867-bib-0272] Gold PB, Macias C, Rodican CF. Does competitive work improve quality of life for adults with severe mental illness? Evidence from a randomized trial of supported employment. Journal of Behavioral Health Services and Research 2014 Feb 7 [Epub ahead of print]. [DOI] [PMC free article] [PubMed]

[CD011867-bib-0273] GRADE Working Group. Handbook for grading the quality of evidence and the strength and recommendations using the GRADE approach. GRADE Working Group, 2013. [Google Scholar]

[CD011867-bib-0274] Hatfield B, Huxley P, Mohamad H. Accommodation and employment: a survey into the circumstances and expressed needs of users of mental health services in a northern town. British Journal of Social Work 1992;22(1):61‐73. [Google Scholar]

[CD011867-bib-0275] Heffernan J, Pilkington P. Supported employment for persons with mental illness: systematic review of the effectiveness of individual placement and support in the UK. Journal of Mental Health 2011;20(4):368‐80. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0276] Higgins JPT, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta‐analyses. BMJ 2003;327:557‐60. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD011867-bib-0277] Higgins JPT, Altman DG, Sterne JAC (editors). Chapter 8: Assessing risk of bias in included studies. In: Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 (updated March 2011). The Cochrane Collaboration, 2011. Available from handbook.cochrane.org.

[CD011867-bib-0278] Higgins JPT, Deeks JJ, Altman DG (editors). Chapter 16: Special topics in statistics. In: Higgins JPT, Green S (editors), Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 (updated March 2011). The Cochrane Collaboration, 2011. Available from handbook.cochrane.org.

[CD011867-bib-0279] Higgins JPT, Jackson D, Barrett JK, Lu G, Ades AE, White IR. Consistency and inconsistency in network meta‐analysis: concepts and models for multi‐arm studies. Research Synthesis Methods 2012;3(2):98‐110. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD011867-bib-0280] Hogarty GE, McEvoy JP, Ulrich RF, DiBarry AL, Bartone P, Cooley S, et al. Pharmacotherapy of impaired affect in recovering schizophrenic patients. Archives of General Psychiatry 1995;52(1):29‐41. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0281] Hsu TH, Ososkie J, Huang YT. Challenges in transition from sheltered workshop to competitive employment: perspectives of Taiwan social enterprise transition specialists. The Journal of Rehabilitation 2009;75(4):19. [Google Scholar]

[CD011867-bib-0282] Kay SR, Fiszbein A, Opfer LA. The positive and negative syndrome scale (PANSS) for schizophrenia. Schizophrenia Bulletin 1987;13(2):261. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0283] Keshavan MS, Vonogradov S, Rumsey J, Sherrill J, Wagner A. Cognitive training in mental disorders: update and future directions. American Journal of Psychiatry 2014;171(5):510‐22. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD011867-bib-0284] Kinoshita Y, Furukawa TA, Kinoshita K, Honyashiki M, Omori IM, Marshall M, et al. Supported employment for adults with severe mental illness. Cochrane Database of Systematic Reviews 2013, Issue 9. [DOI: 10.1002/14651858.CD008297.pub2] [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD011867-bib-0285] Kopelowicz A, Liberman RP, Zarate R. Recent advances in social skills training for schizophrenia. Schizophrenia Bulletin 2006;32(1):S12‐23. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD011867-bib-0286] Krainski J. Sheltered Employment. Encyclopedia of Autism Spectrum Disorders. New York: Springer, 2013:2855‐6. [DOI: 10.1007/978-1-4419-1698-3_1803] [DOI] [Google Scholar]

[CD011867-bib-0287] Kurtz MM, Mueser KT. A meta‐analysis of controlled research on social skills training for schizophrenia. Journal of Consulting and Clinical Psychology 2008;76(3):491‐504. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0288] Latimer E. Community‐based care for people with severe mental illness in Canada. International Journal of Law and Psychiatry 2005;28:561–73. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0289] Lau AL, Cummins RA, McPherson W. An investigation into the cross‐cultural equivalence of the Personal Wellbeing Index. Social Indicators Research 2005;72(3):403‐30. [Google Scholar]

[CD011867-bib-0290] Lehman AF. A quality of life interview for the chronically mentally ill. Evaluation and Program Planning 1988;11(1):51‐62. [Google Scholar]

[CD011867-bib-0291] Lipman RS. Differentiating anxiety and depression in anxiety disorders: use of rating scales. Psychopharmacology Bulletin 1982;18(4):69. [PubMed] [Google Scholar]

[CD011867-bib-0292] Loveland D, Driscoll H, Boyle M. Enhancing supported employment services for individuals with a serious mental illness: a review of the literature. Journal of Vocational Rehabilitation 2007;27(3):177‐89. [Google Scholar]

[CD011867-bib-0293] Lötters F, Carlier B, Bakker B, Borgers N, Schuring M, Burdorf A. The influence of perceived health on labour participation among long term unemployed. Journal of Occupational Rehabilitation 2013;23(2):300‐8. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0294] Marshall M, Lockwood A. Assertive community treatment for people with severe mental disorders. Cochrane Database of Systematic Reviews 1998, Issue 2. [DOI: 10.1002/14651858.CD001089.pub2] [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0295] Marwaha S, Johnson S, Bebbington P, Stafford M, Angermeyer MC, Brugha T, et al. Rates and correlates of employment in people with schizophrenia in the UK, France and Germany. The British Journal of Psychiatry 2007;191:30‐7. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0296] McGurk SR, Mueser KT. Cognitive functioning, symptoms, and work in supported employment: a review and heuristic model. Schizophrenia Research 2004;70(2):147‐73. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0297] McGurk SR, Mueser KT, Pascaris A. Cognitive training and supported employment for persons with severe mental illness: one‐year results from a randomized controlled trial. Schizophrenia Bulletin 2005;31(4):898‐909. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0298] McGurk SR, Mueser KT. Cognitive remediation and supported employment. In: Bukley PF, Gaughran F editor(s). Treatment‐Refractory Schizophrenia ‐ a Clinical Conundrum. Berlin Heidelberg: Springer‐Verlag, 2014:174. [ISBN 978‐3‐64245256‐7] [Google Scholar]

[CD011867-bib-0299] McQuilken M, Zahniser JH, Novak J, Starks, RD, Olmos A, et al. The work project survey: consumer perspectives on work. Journal of Vocational Rehabilitation 2003;18(1):59‐68. [Google Scholar]

[CD011867-bib-0300] Migliore A. Sheltered Workshops. In: JH Stone, M Blouin, editors. International Encyclopedia of Rehabilitation 2010. [http://cirrie.buffalo.edu/encyclopedia/en/article/136/]

[CD011867-bib-0301] Modini M, Tan L, Brinchmann B, Wang MJ, Killackey E, Glozier N, et al. Supported employment for people with severe mental illness: systematic review and meta‐analysis of the international evidence. British Journal of Psychiatry 2016;209(1):14‐22. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0302] Mueser KT, Salyers MP, Mueser PR. A prospective analysis of work in schizophrenia. Schizophrenia Bulletin 2001;27(2):281‐96. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0303] Mueser KT, McGurk SR. Supported employment for persons with serious mental illness: current status and future directions. L'Encéphale 2014;40:S45‐S56. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0304] Norman, C. The Fountain House movement, an alternative rehabilitation model for people with mental health problems, members’ descriptions of what works. Scandinavian Journal of Caring Sciences 2006;20(2):184‐92. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0305] OECD. Sick on the job? Myths and realities about mental health and work. Mental Health and Work. OECD Publishing, 2012. [Google Scholar]

[CD011867-bib-0306] Oliver JPJ, Huxley PJ, Priebe S, Kaiser W. Measuring the quality of life of severely mentally ill people using the Lancashire Quality of Life Profile. Social psychiatry and psychiatric epidemiology 1997;32(2):76‐83. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0307] Overall JE, Gorham DR. The Brief Psychiatric Rating Scale. Psychological reports 1962;10(3):799‐812. [Google Scholar]

[CD011867-bib-0308] Perkins D, Raines J, Tschopp M, Warner T. Gainful employment reduces stigma toward people recovering from schizophrenia. Community Mental Health Journal 2009;45(3):158‐62. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0309] Priebe S, Huxley P, Knight S, Evans S. Application and results of the Manchester Short Assessment of Quality of Life. International Journal of Social Psychiatry 1999;45:7‐12. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0310] Raskin A, Schulterbrandt J, Reatig N, McKeon JJ. Replication of factors of psychopathology in interview, ward behavior and self‐report ratings of hospitalized depressives. The Journal of Nervous and Mental Disease 1969;148(1):87‐98. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0311] Nordic Cochrane Centre, The Cochrane Collaboration. Review Manager 5 (RevMan) 5. Version 5.3. Nordic Cochrane Centre, The Cochrane Collaboration, 2014.

[CD011867-bib-0312] Ruggeri M, Leese M, Thornicroft G, Bisoffi G, Tansella M. Definition and prevalence of severe and persistent mental illness. The British Journal of Psychiatry 2000;177(2):149‐55. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0313] Salanti G, Ades AE, Ioannidis JP. Graphical methods and numerical summaries for presenting results from multiple‐treatment meta‐analysis: an overview and tutorial. Journal of Clinical Epidemiology 2011;64(2):163‐71. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0314] Salanti G, Giovane C, Chaimani A, Caldwell DM, Higgins JP. Evaluating the quality of evidence from a network meta‐analysis. PloS One 2014;9(7):e99682. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD011867-bib-0315] Salkever DS, Karakus MC, Slade EP, Harding CM, Hough RL, Rosenheck RA, et al. Measures and predictors of community‐based employment and earnings of persons with schizophrenia in a multisite study. Psychiatric Services 2007;58(3):315‐24. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0316] Savio M, Righetti A. Cooperatives as a social enterprise in Italy: a place for social integration and rehabilitation. Acta Psychiatrica Scandinavica 1993;88:238‐42. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0317] Schandelmaier S, Ebrahim S, Burkhardt SC, Boer WE, Zumbrunn T, Guyatt, et al. Return to work coordination programmes for work disability: a meta‐analysis of randomised controlled trials. PLoS One 2012;7(11):e49760. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD011867-bib-0318] Schuring M, Robroek SJ, Otten FW, Arts CH, Burdorf A. The effect of ill health and socioeconomic status on labor force exit and re‐employment: a prospective study with ten years follow‐up in the Netherlands. Scandinavian Journal of Work, Environment and Health 2013;39(2):134‐43. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0319] Stein LI, Alberto BS. Assertive Community Treatment of persons with severe mental illness. New York: WW Norton & Co, 1998. [Google Scholar]

[CD011867-bib-0320] Svanberg E, Gumley A, Wilson A. How do social firms contribute to recovery from mental illness? A qualitative study. Clinical Psychology and Psychotherapy 2010;17(6):482–96. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0321] Tsang HW, Chan A, Wong A, Liberman RP. Vocational outcomes of an integrated supported employment program for individuals with persistent and severe mental illness. Journal of Behavior Therapy and Experimental Psychiatry 2009;40(2):292‐305. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0322] Twamley EW, Jeste DV, Lehman AF. Vocational rehabilitation in schizophrenia and other psychotic disorders: a literature review and meta‐analysis of randomized controlled trials. Journal of Nervous and Mental Disease 2003;191(8):515‐23. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0323] Villotti P, Zaniboni S, Fraccaroli F. Social cooperatives in Italy [Les entreprises à économie sociale en Italie]. L'Encéphale 2014;40:S57‐S65. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0324] Ware JE Jr, Kosinski M, Keller SD. A 12‐item short‐form health survey: construction of scales and preliminary tests of reliability and validity. Medical Care 1996;34(3):22‐233. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0325] White IR. Multivariate random‐effects meta‐regression: updates to mvmeta. The Stata Journal 2011;11(2):255‐70. [Google Scholar]

[CD011867-bib-0326] White IR, Barrett JK, Jackson D, Higgins JPT. Consistency and inconsistency in network meta‐analysis: model estimation using multivariate meta‐regression. Research Synthesis Methods 2012;3:111‐25. [DOI] [PMC free article] [PubMed] [Google Scholar]

[CD011867-bib-0327] Wing JK. A simple and reliable subclassification of chronic schizophrenia. The British Journal of Psychiatry 1961;107(450):862‐75. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0328] Wykes T, Huddy V, Cellard C, McGurk SR, Czobor P. A meta‐analysis of cognitive remediation for schizophrenia: methodology and effect sizes. American Journal of Psychiatry 2011;168(5):472‐85. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0329] Zigmond AS, Snaith RP. The hospital anxiety and depression scale. Acta Psychiatrica Scandinavica 1983;67(6):361‐70. [DOI] [PubMed] [Google Scholar]

[CD011867-bib-0330] Suijkerbuijk YB, Verbeek JH, Mechelen J, Ojajärvi A, Anema JR, Corbiere M, et al. Interventions for obtaining and maintaining employment in adults with severe mental illness, a network meta‐analysis. Cochrane Database of Systematic Reviews 2015, Issue 9. [DOI: 10.1002/14651858.CD011867] [DOI] [PMC free article] [PubMed] [Google Scholar]

PERMALINK

Interventions for obtaining and maintaining employment in adults with severe mental illness, a network meta‐analysis

Yvonne B Suijkerbuijk

Frederieke G Schaafsma

Joost C van Mechelen

Anneli Ojajärvi

Marc Corbière

Johannes R Anema

Abstract

Background

Objectives

Search methods

Selection criteria

Data collection and analysis

Main results

Authors' conclusions

Plain language summary

Summary of findings

Summary of findings for the main comparison. Summary of findings of network meta‐analysis.

Background

Description of the condition

Description of the intervention

Prevocational training

Social skills training

Cognitive training

Transitional employment

Sheltered workshop

Social enterprise

The Clubhouse model

Supported employment

Individual placement and support

Augmented supported employment

Psychiatric care only

Assertive community treatment

How the intervention might work

Why it is important to do this review

Objectives

Methods

Criteria for considering studies for this review

Types of studies

Types of participants

Types of interventions

Prevocational training

Transitional employment

Supported employment

Augmented supported employment

Psychiatric care only

Types of outcome measures

Primary outcomes

Percentage or number of participants who obtained competitive employment

Secondary outcomes

Employment

Clinical outcomes

Adverse events

Search methods for identification of studies

Electronic searches

Searching other resources

Data collection and analysis

Selection of studies

Data extraction and management

Assessment of risk of bias in included studies

Measures of treatment effect

Unit of analysis issues

Cluster‐RCTs

Multi‐arm studies

Dealing with missing data

Assessment of heterogeneity

Assessment of reporting biases

Data synthesis

Relative treatment effects

Methods for direct and indirect or mixed treatment comparisons

Assessment of statistical inconsistency in network meta‐analysis

Relative treatment ranking

Subgroup analysis and investigation of heterogeneity

Sensitivity analysis

Quality of the evidence

Results

Description of studies

Results of the search

1.