Abstract
Objective
To determine the most effective way of helping people with severe mental illness to obtain competitive employment—that is, a job paid at the market rate, and for which anyone can apply.
Design
Systematic review.
Participants
Eligible studies were randomised controlled trials comparing prevocational training or supported employment (for people with severe mental illness) with each other or with standard community care.
Outcome measures
The primary outcome was number of subjects in competitive employment. Secondary outcomes were other employment outcomes, clinical outcomes, and costs.
Results
Eleven trials met the inclusion criteria. Five (1204 subjects) compared prevocational training with standard community care, one (256 subjects) compared supported employment with standard community care, and five (484 subjects) compared supported employment with prevocational training. Subjects in supported employment were more likely to be in competitive employment than those who received prevocational training at 4, 6, 9, 12, 15, and 18 months (for example, 34% v 12% at 12 months; number needed to treat 4.45, 95% confidence interval 3.37 to 6.59). This effect was still present, although at a reduced level, after a sensitivity analysis that retained only the highest quality trials (31% v 12%; 5.3, 3.6 to 10.4). People in supported employment earned more and worked more hours per month than those who had had prevocational training.
Conclusion
Supported employment is more effective than prevocational training at helping people with severe mental illness obtain competitive employment.
Introduction
In the United States it is estimated that 75-85% of people with severe mental illness are unemployed,1,2 whereas estimates in the United Kingdom range from 61% to 73%.3,4 Yet despite these high unemployment rates, surveys consistently show that most people with severe mental illness want to work.5,6
There are compelling ethical, social, and clinical reasons for helping people with mental illness to work. From an ethical standpoint, the right to work is enshrined in the Universal Declaration of Human Rights 1948 and has been incorporated into national legislation, such as the UK Disability Discrimination Act 1995. From a social standpoint, high unemployment rates are an index of the social exclusion of people with mental illness, which the US and UK governments, among others, are committed to reducing.7,8 From a clinical standpoint, employment may lead to improvements in outcome through increasing self esteem, alleviating psychiatric symptoms, and reducing dependency.7
Prevocational training and supported employment are different ways of helping people with severe mental illness return to work. Prevocational training assumes that people with severe mental illness require a period of preparation before entering into competitive employment—that is, a job paid at the market rate, and for which anyone can apply. This includes sheltered workshops, transitional employment (working in a job that is “owned” by a rehabilitation agency), work crews, skills training, and other preparatory activities.9
Supported employment places clients in competitive jobs without extended preparation and provides on the job support from trained “job coaches” or employment specialists.10 The core principles of supported employment are that (a) the goal is competitive employment in work settings integrated into a community's economy, (b) clients are expected to obtain jobs directly, rather than after lengthy pre-employment training, (c) rehabilitation is an integral component of treatment of mental health rather than a separate service, (d) services are based on client's preferences and choices, (e) assessment is continuous and based on real work experiences, and (f ) follow on support is continued indefinitely.10,11
In the United States there are about 3000 “psychiatric rehabilitation providers” offering some form of prevocational training, whereas there are more than 36 000 people with mental illness in supported employment schemes.12,13 In the United Kingdom prevocational training is still the norm, but there are at least 80 agencies offering supported employment.14
It is unclear how far prevocational training and supported employment are effective at helping people with severe mental illness to obtain competitive employment. We aimed to evaluate the effectiveness of the two approaches.
Methods
Search strategy and inclusion criteria
We electronically searched CINAHL (1982-98), Embase (1980-98), Medline (1966-98), and PsychLIT (1987-98). The search proceeded by exploding the appropriate index term for mental disorder in each database and combining this with a free text search using (supp* employ*) or (employment) or (psychosocial rehab*) or (psychiatric rehab*) or (occupational rehab*) or (soc* rehab*) or (work rehab*) or (job rehab*) or (sheltered work*) or (transitional employ*) or (rehabilitation counselling) or (vocation*) or (fountain house*) or (fountain-house*) or (clubhouse*) or (club-house*). The results of this search were then combined with a search using the Cochrane Collaboration search string for potential trials and reviews.15 We also carried out a free text search on the collaboration's register of randomised controlled trials. The sensitivity of the search strategy was evaluated by determining how many trials cited in the reference lists of the identified trials and reviews had not been detected. Of three undetected trials, two were not listed on any of the databases, and the third trial was indexed under “delivery of health care/integrated.” A further search using this index term detected no further trials.
Two reviewers (MM and RC) independently assessed the published reports of the identified trials and decided which met the inclusion criteria. These were that the trial was a randomised controlled one, analysed on an intention to treat basis, in which prevocational training or supported employment was compared with standard community care or each other; outcome data were provided on 50% or more of randomised subjects; and most subjects were aged 18 to 65 and had severe mental illness (schizophrenia, bipolar disorder, or depression with psychotic features). Studies meeting inclusion criteria were allocated to one of three comparisons (prevocational training versus community control, supported employment versus community control, or supported employment versus prevocational training). Prevocational training was defined as any approach in which participants were expected to undergo a period of preparation before being encouraged to seek competitive employment. Supported employment was defined as any approach that attempted to place clients immediately in competitive employment (although a short period of orientation was acceptable if of less than one month's duration and not involving sheltered work, employment training, or transitional employment). Interrater agreement was assessed for overall eligibility of trials (κ=0.89) and for allocation of trials to comparisons (κ=1).
Trials were classified according to allocation concealment15 as A, adequate (the method for assigning participants to interventions was robust to patient and clinician bias and clearly described); B, unclear; and C, inadequate (the method of assignment was not robust to patient and clinician bias). When the method was unclear, trialists were contacted for further details.
Blinding of patients and treating clinicians is not possible in trials of vocational rehabilitation. It is also difficult for those evaluating outcome to remain blind to group allocation, as they are obliged to collect data that indicates group allocation (for example, days in different types of employment). The trials were, however, rated on independence of evaluators from those providing the intervention.
Data extraction and analysis
The primary outcome was number of clients in competitive employment at various times. Other secondary employment outcomes were number of subjects in any form of employment (including transitional, sheltered, or voluntary work), mean hours per month in competitive employment, and mean monthly earnings. In addition data were extracted on clinical and social outcome (including number of people participating and number admitted to hospital) and costs (mean monthly costs of the programme and of all health care). Categorical data and continuous data were extracted independently by MM and RC and cross checked by double entry. Continuous data were excluded if collected using an unpublished scale or based on a subset of items from a scale (such data are known to be biased in psychiatric trials).16 For categorical data we calculated the relative risk with confidence intervals. The number needed to treat for one person to obtain competitive employment was calculated as the inverse of the absolute risk reduction for being unemployed. Confidence intervals for the number needed to treat were calculated using the Arcus Quickstat program (Research Solutions, Cambridge).
Heterogeneity was defined as a significance level of 0.1 by the χ2 test for heterogeneity. Where heterogeneity was present the data were reanalysed using a random effects model. We conducted a sensitivity analysis, excluding trials with allocation concealment in categories B or C, non-independent evaluators, or follow up rates of less than 75%.
Results
We identified 40 trials and 13 reviews. We excluded 29 trials because the trial was not randomised (11 trials), the participants did not have severe mental illness (3), the intervention did not involve vocational rehabilitation (6), the number of participants was unclear (2), the trial compared a modification of prevocational training with unmodified prevocational training (4), and the trial compared prevocational training with continuing care in hospital (3). Eleven trials met the inclusion criteria (see tables on website) and were allocated to comparisons of prevocational training with standard care (five trials; 1204 subjects), supported employment with standard care (one trial; 256), and supported employment with prevocational training (five trials; 491).
Prevocational training versus standard care
Of the five trials comparing prevocational training with standard care, one had adequate allocation concealment, two used independent evaluators, and four had follow up rates of greater than 75%. No trial was eligible for the sensitivity analysis. Two trials provided data on the primary outcome of number of subjects in competitive employment, but these showed no evidence that prevocational training was superior to control (18 months (28 subjects): relative risk 1.18, 95% confidence interval 0.87 to 1.61; 24 months (215): 0.95, 0.77 to 1.17).17,19 Three trials reported data on number of subjects in any form of employment, showing no evidence that prevocational training was superior to control at 3, 6, 9, 12, and 18 months.17,18,20 Two trials found no difference in the number of clients participating in the programme between prevocational training and control groups (284 subjects; relative risk 0.97, 0.73 to 1.30).18,20 Three trials showed that significantly fewer patients were admitted to hospital among those receiving prevocational training (887 subjects; 0.79, 0.65 to 0.95).17,18,21 Heterogeneity was present in this outcome, and on reanalysis using a random effects model the difference failed to reach significance (0.71, 0.48 to 1.04). One trial reported no significant difference in self esteem between prevocational training and control groups (28 subjects; 25.5 (SD 6.6) and 23.3 (7.3), respectively).19 One trial reported mean monthly total healthcare costs of $417.90 (£292.83) for the prevocational training group and $651.50 (£456.52) for controls, but no statistical analysis was reported.18
Supported employment versus standard care
Only one trial provided data for supported employment compared with standard care.22 Although the trial used independent raters, the method of allocation concealment was unclear and the follow up rate was only 71%. A further problem was that the intervention combined supported employment with assertive community treatment, whereas the control was standard community care. For 256 subjects there was no difference in those in competitive employment between supported employment and control at 12 months (relative risk 1.01, 0.93 to 1.09), but there was a significant difference favouring supported employment at 24 months (0.92, 0.85 to 0.99) and 36 months (0.88, 0.82 to 0.96). Clients receiving supported employment were more likely to be in any form of employment at 12 months (0.79, 0.70 to 0.90; number needed to treat 5.5) and to earn more per month (supported employment group $60.50 (£42.39), control group $26.90 (£18.85); P<0.05). Participation rates and number of hospital admissions were not significantly different between clients receiving supported employment and controls (0.74, 0.55 to 1.01 and 0.83, 0.63 to 1.10, respectively). Mean monthly healthcare costs were significantly higher for clients in the supported employment group ($1599.00 (£1120.45) versus $527.30 (£369.49) for controls), but this finding is difficult to interpret as clients receiving supported employment also received assertive community treatment.
Supported employment versus prevocational training
Of the five trials comparing supported employment with prevocational training, four had adequate allocation concealment, four used independent evaluators, and all had follow up rates greater than 75%. In one trial, however, the intervention combined supported employment with assertive community treatment, whereas the control was standard community care.27 Data from the five trials showed a significant difference in favour of supported employment at 4, 6, 9, 12, 15, and 18 months for those likely to be in competitive employment (for example, at 12 months 34% in supported employment and 12% who received prevocational training, relative risk 0.76, 0.69 to 0.84; number needed to treat 4.45, 3.37 to 6.59 (figure)). Heterogeneity was present at 12 months, but the difference in favour of supported employment remained significant after reanalysis using a random effects model (0.76, 0.64 to 0.89). One trial reported the number of subjects in any form of employment, finding no significant difference between supported employment and prevocational training at 6, 12, and 18 months.27 Three trials found that clients in supported employment had significantly more hours per month in competitive employment than those who received prevocational training (table 1). Three of four trials found that clients in supported employment had higher mean monthly earnings that those who received prevocational training (table 1). There were insufficient data to determine whether there was a difference in participation rates between supported employment and prevocational training at 6, 12, and 18 months. Two trials reported data on self esteem, quality of life, and severity of symptoms but found no significant differences.24,25 One trial reported that programme costs of supported employment were greater than those for prevocational training, but that overall healthcare costs were less for people in supported employment.23 Another trial found no significant difference in programme costs and overall healthcare costs.24
Table 1.
Other employment outcomes for supported employment versus prevocational training
| Study | Mean (SD) hours worked per month | Mean (SD) monthly earnings ($) |
|---|---|---|
| Bond et al23 | Not known | 127.1 (190) v 71.7 (234)*, P<0.05 |
| Drake et al24 | 33.7 (46) v 11.4 (22), P<0.001 | 188.5 (302) v 59.9 (124)†, P=0.001 |
| Drake et al25 | 17.9 (31) v 1.5 (6.9), P<0.001 | 111.1 (188) v 111.4 (163)‡, NS but no exact P value reported |
| Gervey and Bedell26 | 69.0 v 9.9 (no SD reported), P<0.03 | Not reported |
| McFarlane et al27 | Not known | 41.9 v 11.8 (no SD reported)§, P=0.019 (Mann-Whitney) |
Conversion factor at time of writing: *£89.06 v £50.24; †£132.09 v £41.97; ‡£77.85 v £78.06; §£29.36 v £8.27.
We conducted a sensitivity analysis on the primary outcome of number of subjects in competitive employment, excluding trials with allocation concealment in categories B or C, those that used non-independent evaluators, or those with follow up rates of less than 75%. This left data from three trials,24,25,27 which continued to show a significant difference in favour of supported employment at 4, 6, 9, 12, 15, and 18 months. This difference remained after exclusion of the trial that combined supported employment with assertive community treatment (for example, at 12 months 31% in supported employment and 12% who received prevocational training; number needed to treat 5.3, 3.6 to 10.4).27
Discussion
Supported employment was more effective than prevocational training at helping people with severe mental illness to obtain competitive employment. This finding was robust to a sensitivity analysis that excluded all but the two highest quality trials and was supported by data for other employment outcomes. Data on clinical and social functioning and costs were inconclusive but suggested no major differences between supported employment and prevocational training. The five trials of supported employment versus prevocational training showed good recruitment of women, people from ethnic minorities, and people with schizophrenia (table 2), which suggests that the main finding of the review can be applied to the general population of patients with severe mental illness who desire to work. Generalisability is, however, limited by the fact that all the trials were conducted in the United States. It remains uncertain whether supported employment will be more effective than prevocational training in countries with less dynamic economies and dissimilar welfare structures.
Table 2.
Characteristics of trial participants. Values are percentages unless stated otherwise
| Trial | Age (years) | Female | Member of ethnic minority | Schizophrenic | Married |
|---|---|---|---|---|---|
| Bond et al23 | 35 | 49 | 20 | 66 | 52 |
| McFarlane et al27 | 32.9 | 30.4 | 7 | 65.1 | 26 |
| Drake et al24 | 37 | 51.7 | 5 | 47 | 49.7 |
| Drake et al25 | 39.4 | 61.2 | 82.9 | 67 | 34.2 |
| Gervey and Bedell26 | 19 | 33 | 83 | “Severe mental disorder” | Not known |
| Weighted mean | 35.5 | 49.8 | 37.9 | 60.2 | 41.3 |
Only one trial compared supported employment with standard community care. Although this trial suggested that supported employment was superior to standard community care, its findings are difficult to interpret as the group receiving supported employment also received assertive community treatment.
The included trials of prevocational training compared with standard community care were of limited quality, and none met the criteria for the sensitivity analysis. The data available from these trials were insufficient to make judgments on the effectiveness of prevocational training over standard community care. Only two of five trials in this comparison reported data on the primary outcome of competitive employment. This omission may reflect selective reporting of results. Interestingly, clients receiving prevocational training were significantly less likely to be admitted to hospital than those receiving standard community care, but there was heterogeneity with this outcome, and the finding was not significant when analysed by a random effects model. Trials of supported employment did not usually report data on hospital admissions.
With the passing of the Disability Discrimination Act 1995, the UK government signalled its commitment to helping disabled people return to the workplace. People disabled by severe mental illness have particularly high unemployment rates. Our review indicates that supported employment is a more effective way of helping such people find competitive employment than is prevocational training. The UK government should therefore encourage agencies concerned with vocational rehabilitation to develop and evaluate supported employment schemes similar to those in the United States.
What is already known on this topic
Prevocational training (a period of preparation before entering competitive employment) and supported employment (placement in competitive employment while offering on the job support) are ways of helping people with severe mental illness obtain work
Both methods are widely practised, but it is unclear which is most effective, and so far the available data have not been subject to a systematic review and meta-analysis
What this study adds
Supported employment is more effective than prevocational training at helping people with severe mental illness to obtain and keep competitive employment
Supplementary Material
Figure.
Number of subjects in competitive employment after supported employment or prevocational training
Footnotes
Funding: The research was supported by the NHS Health Technology Assessment Program (grant number 96/41/3). The views expressed in this paper are not necessarily those of this programme.
Competing interests: GRB has a close collaborative relationship with Bob Drake and Debbie Becker, developers of the individual placement and support model.
Details of the trials are available on the BMJ's website
References
- 1.Lehman AF. Vocational rehabilitation in schizophrenia. Schizophr Bull. 1995;21:645–656. doi: 10.1093/schbul/21.4.645. [DOI] [PubMed] [Google Scholar]
- 2.Ridgeway P, Rapp C. The active ingredients in achieving competitive employment for people with psychiatric disabilities: a research synthesis. Lawrence, KS: Commission on Mental Health and Developmental Disabilities; 1998. . (Critical Ingredients series.) [Google Scholar]
- 3.Meltzer H, Gill B, Petticrew M, Hinds K. Economic activity and social functioning of adults with psychiatric disorders. London: HMSO; 1995. . (Office of Population Censuses and Surveys, Surveys of psychiatric morbidity in Great Britain report 2.) [Google Scholar]
- 4.McCreadie RG. The Nithsdale schizophrenia surveys. Soc Psychiatry Psychiatr Epidemiol. 1992;27:40–45. doi: 10.1007/BF00788955. [DOI] [PubMed] [Google Scholar]
- 5.Shepherd G, Murray A, Muijen M. Relative values: the different views of users, family carers and professionals on services for people with schizophrenia. London: Sainsbury Centre for Mental Health; 1994. [Google Scholar]
- 6.Hatfield B, Huxley P, Mohamad H. Accommodation and employment: a survey into the circumstances and expressed needs of users of mental health services in a northern town. Br J Soc Work. 1992;22:60–73. [Google Scholar]
- 7.Cook A, Razzano L. Vocational rehabilitation for persons with schizophrenia: recent research and implications for practice. Schizophr Bull. 2000;26:87–103. doi: 10.1093/oxfordjournals.schbul.a033448. [DOI] [PubMed] [Google Scholar]
- 8.Department of Health. Modernising mental health services. London: DoH; 1998. [Google Scholar]
- 9.Bond GR. Vocational rehabilitation. In: Liberman RP, editor. Handbook of psychiatric rehabilitation. New York: Macmillan; 1992. pp. 244–275. [Google Scholar]
- 10.Becker DR, Drake RE, Concord NH. Individual placement and support: a community mental health center approach to vocational rehabilitation. Community Ment Health J. 1994;30:193–206. doi: 10.1007/BF02188630. [DOI] [PubMed] [Google Scholar]
- 11.Bond GR, Drake RE, Mueser KT, Becker DR. An update on supported employment for people with severe mental illness. Psychiatric Serv. 1997;48:335–346. doi: 10.1176/ps.48.3.335. [DOI] [PubMed] [Google Scholar]
- 12.Bond GR, Drake RE, Becker DR, Mueser KT. Effectiveness of psychiatric rehabilitation approaches for employment of people with severe mental illness. J Disab Policy Stud. 1999;10:18–52. [Google Scholar]
- 13.Wehman P, Revell WG, Kregel J. Supported employment: a decade of rapid growth and impact. In: Weham P, Revell WG, West M, editors. Supported employment research: expanding competitive employment opportunities for persons with significant disabilities. Richmond, VA: VCU Rehabilitation Research and Training Center on Supported Employment; 1997. pp. 1–18. [Google Scholar]
- 14.ERMIS Database. Dorking: European Economic Interest Grouping; 1998. [Google Scholar]
- 15.Clark M, Oxman AD. The Cochrane Collaboration reviewers' handbook version 4.0. Oxford: Cochrane Collaboration; 1999. [Google Scholar]
- 16.Marshall M, Lockwood L, Bradley C, Adams C, Joy C, Fenton M. Unpublished rating scales—a major source of bias in randomised controlled trials of treatments for schizophrenia. Br J Psychiatry. 2000;176:249–252. doi: 10.1192/bjp.176.3.249. [DOI] [PubMed] [Google Scholar]
- 17.Beard JH, Pitt RB, Fisher SH, Goertzel V. Evaluating the effectiveness of a psychiatric rehabilitation program. Am J Orthopsychiatry. 1963;33:701–712. doi: 10.1111/j.1939-0025.1963.tb01018.x. [DOI] [PubMed] [Google Scholar]
- 18.Dincin J, Witheridge TF. Psychiatric rehabilitation as a deterrent to recidivism. Hosp Community Psychiatry. 1982;33:645–650. doi: 10.1176/ps.33.8.645. [DOI] [PubMed] [Google Scholar]
- 19.Griffiths RD. Rehabilitation of chronic psychotic patients. Psychol Med. 1974;4:316–325. [PubMed] [Google Scholar]
- 20.Okpaku SO, Anderson KH, Sibulkin AE, Butler JS, Bickman L. The effectiveness of a multidisciplinary case management team on the employment of SSDI applicants and beneficiaries. Psychiatr Rehabil J. 1997;20:34–41. [Google Scholar]
- 21.Wolkon GH, Karmen M, Tanaka HT. Evaluation of a social rehabilitation program for recently released psychiatric patients. Community Ment Health J. 1971;7:312–322. doi: 10.1007/BF01434441. [DOI] [PubMed] [Google Scholar]
- 22.Chandler D, Meisel J, Hu T, McGowen M, Mintz J, Madison K. Client outcomes in a three year controlled study of an integrated service model agency. Psychiatr Serv. 1996;47:1337–1343. doi: 10.1176/ps.47.12.1337. [DOI] [PubMed] [Google Scholar]
- 23.Bond GR, Dietzen LL, McGrew JH, Miller LD. Accelerating entry into supported employment for persons with severe psychiatric disabilities. Rehabil Psychol. 1995;40:75–94. [Google Scholar]
- 24.Drake RE, McHugo GJ, Becker DR, Anthony WA, Clark RE. The New Hampshire study of supported employment for people with severe mental illness. J Consult Clin Psychol. 1996;64:391–399. doi: 10.1037//0022-006x.64.2.391. [DOI] [PubMed] [Google Scholar]
- 25.Drake RE, McHugo GJ, Bebout DR, Harris M, Bond GR, Quimby EA. A randomized controlled trial of supported employment for inner-city patients with severe mental illness. Arch Gen Psychiatry. 1999;56:627–633. doi: 10.1001/archpsyc.56.7.627. [DOI] [PubMed] [Google Scholar]
- 26.Gervey R, Bedell JR. Supported employment in vocational rehabilitation. In: Bedell JR, editor. Psychological assessment and treatment of persons with severe mental disorders. Washington, DC: Taylor and Francis; 1994. [Google Scholar]
- 27.McFarlane WR, Dushay RA, Deakins SM, Stasny P, Lukens EP, Toran J, et al. Employment outcomes in family-aided assertive community treatment. Am J Orthopsychiatry. 2000;70:203–214. doi: 10.1037/h0087819. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.