Supported employment for adults with severe mental illness

Abstract

This is the protocol for a review and there is no abstract. The objectives are as follows:

1. To review the effectiveness of supported employment compared to other approached to vocational rehabilitation and treatment as usual.

2. Secondary objectives are to establish how far:

   1. fidelity to the IPS model affects the effectiveness of supported employment,
   2. the effectiveness of supported employment can be augmented by the addition of other interventions.

BACKGROUND

Description of the condition

People who suffer from severe mental disorder experience high rates of unemployment. A review of eight controlled trials demonstrated that employment rates for people with schizophrenia, even with optimal support, ranged from only 30% to 80%, with a median of 60% across these studies (Bond 2004). These low employment rates reflect the disability caused by severe mental illness, but they may also reflect discrimination (unemployment rates are higher than in other disabled groups) (ONS 1998) and the low priority given to employment by psychiatric services (Lehman 1995). Despite high unemployment rates amongst the severely mentally ill, surveys have consistently shown that most want to work (Hatfield 1992; Lehman 1995; Shepherd 1994). Mental health issues said to be linked to unemployment include: cognitive impairment, psychotic symptoms, negative symptoms, fear of losing benefits, stigma and lack of access to employment services (Bond 1991; Bond 2008b; Cook 2006; Rosenheck 2006; Rutman 1994).

Description of the intervention

Supported employment is an approach to vocational rehabilitation that involves trying to place clients in competitive jobs without any extended preparation (Bond 1992). Originally developed for people with learning disabilities, supported employment has been defined as ‘paid work that takes place in normal work settings with provision for ongoing support services’ (Becker 1994; Bond 1999). Proponents of supported employment had two objections to pre-vocational training, which adheres to the key principle that a period of preparation is necessary before entering competitive employment (Bilby 1992; Bond 1997a). First, they argued that it promoted dependency and deterred clients from finding competitive employment. Second, they argued that pre-vocational training was not effective in developing work skills. Instead of pre-vocational training, they proposed trying to place clients as quickly as possible in competitive employment positions, where they would receive intensive on-the-job support and training from personnel known as ‘job coaches’ (Anthony 1987).

Individual placement and support (IPS) model is a carefully specified form of supported employment which is based on close adherence to seven key principles (Mueser 2004). These principles are: (a) the goal is competitive employment in work settings integrated into a community’s economy, (b) services are based on clients’ choices, (c) clients are expected to obtain jobs directly, rather than following lengthy pre-employment training (rapid job search), (d) attention to patient preference in the job search, (e) integration between employment services and mental health treatment teams, (f) ongoing individual support, and (g) systematic benefits counselling (Bond 2008a). Adherence to individual placement and support guidelines may be measured using a fidelity scale (Bond 1997b). In IPS, employment specialists serve on clients’ treatment teams alongside other staff, such as case managers and psychiatrists. Each employment specialist provides the full range of vocational services to each client, including engagement in services, identifying job interests and vocational assessment, job finding, and job support. IPS uses assertive outreach (Stein 1998) to deliver vocational services in the community rather than at mental health or rehabilitation agencies (Bond 1997b).

How the intervention might work

Supported employment is defined as ‘paid work that takes place in normal work settings with provision for ongoing support services’ (Becker 1994; Bond 1999). It helps people with mental illness to work by placing them as quickly as possible in competitive employment positions, where they would receive intensive on-the-job support and training from personnel known as job coaches (Anthony 1987).

Why it is important to do this review

A previous Cochrane review (Crowther 2001) and another systematic review (Twamley 2003) has examined effectiveness of various types of vocational rehabilitation for individuals with severe mental illness, including supported employment, but it now requires updating as several new trials of supported employment have been published recently. These trials have been covered in two narrative reviews but there has been no formal meta-analytic summaries as yet (Bond 2004; Bond 2008a).

OBJECTIVES

1. To review the effectiveness of supported employment compared to other approached to vocational rehabilitation and treatment as usual.

2. Secondary objectives are to establish how far:

   1. fidelity to the IPS model affects the effectiveness of supported employment,
   2. the effectiveness of supported employment can be augmented by the addition of other interventions.

METHOD

Criteria for considering studies for this review

Types of studies

Randomised controlled trials (RCT) that assess the effects of supported employment in people with severe mental illness. Quasi-randomised studies, such as those allocating by using alternate days of the week, will be excluded.

Types of participants

The supported employment was not designed for a specific diagnostic group nor was it applied in a diagnostic-specific way in everyday practice. Therefore, for the purpose of this review, the main requirements of participants are that they are similar to those who typically present to the supported employment services. Specific inclusion criteria are that a majority of clients in the trial are (a) of working age (normally 16 to 70 years); (b) unemployed; and (c) suffering from severe mental illness, defined as: schizophrenia and schizophrenia-like disorders; bipolar disorders; or depression with psychotic features. Substance abuse and post traumatic stress disorder are not considered severe mental illness, but trials will be eligible if participants had a problem with substance abuse and/or comorbidity of post traumatic stress disorder in addition to severe mental illness. Trials will be excluded where a majority of participants (more than 50%) were suffering from a learning disability as the sole psychiatric diagnosis.

Types of interventions

Three interventions of interest were defined: supported employment (including Individual Placement and Support, and Augmented Supported Employment), other vocational approaches and treatment as usual.

1. Supported employment: Supported employment is a technique designed to help mentally ill people obtain and keep competitive employment. Supported employment aims to help clients obtain competitive work as quickly as possible and provides ongoing support to help them keep their employment (Bond 2001; Mueser 2004).

   • 1.1Individual placement and support (IPS): IPS is a carefully specified approach to supported employment which requires close adherence to the seven principles described above. Fidelity to the IPS model can be assessed using an IPS Fidelity Scale (Becker 2001). IPS will be classified into two categories (i) Low fidelity IPS and (ii) High fidelity IPS. The seven key principles described in Description of the intervention are taken into consideration to assess the fidelity (Bond 1997b). Low fidelity IPS is defined as a) the programme itself does not satisfy one or more of the seven key principles, for example if the same personnel is in charge of employment service and clinical service; b) although the programme does satisfy all the seven criteria, the quality assessment reveals that the actual delivery of the programme did not satisfy one or more of the seven key principles; or c) the quality of the actual delivery was not assessed. High fidelity IPS is when the programme satisfies all the seven criteria.
   • 1.2Augmented supported employment: Supported employment can be augmented with other interventions, such as motivational interventions, social skills training and cognitive rehabilitation (Drake 2008; Bell 2008; McGurk 2007; Mueser 2005; Tsang 2007; Wallace 2004).

2. Other vocational approaches

   Other vocational approaches are described in detail in the current version of the review (Crowther 2001) and include sheltered work-shop; prevocational training classes; job counselling; and Clubhouse model - this model provides (a) work experiences through clubhouse work units, (b) transitional employment (the participant works for a limited period in a paid position in a real workplace, but the position is “owned” by the employment agency rather than the participant) and (c) peer support. In this model, the participant graduates from helping to maintain a patient-led “clubhouse”, to transitional employment, and finally to competitive employment; and diversified placement approach - principles of this approach are (a) goal of paid employment including but not limited to competitive one, (b) gradualism (members move gradually through the vocational continuum), (c) flexibility in movement between placements, (d) peer support and (f) partnerships with the business community (Bond 2004). All of these approaches differ from supported employment in that they do not place an emphasis on an immediate search for competitive employment, but prefer a period of preparation, before seeking competitive employment. We will treat them as a single control intervention.

3. Treatment as usual

   Treatment as usual is defined as standard psychiatric care for participants in the trial, without any specific vocational component. It is assumed that both intervention and control participants will be receiving treatment as usual, which would normally include: medication, medication management, case management, and supportive psychotherapy (Bond 2008c).

Types of outcome measures

We will group outcomes into short term (less than six months) medium term (six months to one year) and long term (over one year)

Primary outcomes

1. Employment: days in competitive employment (long term)

Secondary outcomes

1. Employment

   • 1.1Days in competitive employment (medium term)
   • 1.2Days in any form of paid employment (such as competitive employment, transitional employment, or sheltered employment with wage)
   • 1.2Earnings in the first 1 years
   • 1.3Job stability (e.g. number of participants who continuously held competitive employment for more than 90 days)
   • 1.4Time to first competitive employment
   • 1.5Numbers not participating in programmes (as defined by individual studies)

2. Education

   • 2.1Days in any form of employment or education (including training courses or full or part-time education)

3. Leaving the study early

   • 3.1for any reason
   • 3.2specific reason (as defined by individual studies)

4. Global state

   • 4.1Relapse
   • 4.2Time to relapse
   • 4.3No clinically important change in global state
   • 4.4Not any change in global state
   • 4.5Average endpoint global state score
   • 4.6Average change in global state scores

5. Mental state

   • 5.1No clinically important change in general mental state
   • 5.2Not any change in general mental state
   • 5.3Average endpoint general mental state score
   • 5.4Average change in general mental state scores
   • 5.5No clinically important change in specific symptoms
   • 5.6Not any change in specific symptoms
   • 5.7Average endpoint specific symptom score
   • 5.8Average change in specific symptom scores

6. Service Use

   • 6.1Mean days in hospital
   • 6.2Number of participants admitted to hospital/re-hospitalised

7. Quality of life

   • 7.1No clinically important change in quality of life
   • 7.2Not any change in quality of life
   • 7.3Average endpoint quality of life score
   • 7.4Average change in quality of life scores
   • 7.5No clinically important change in specific aspects of quality of life
   • 7.6Not any change in specific aspects of quality of life
   • 7.7Average endpoint specific aspects of quality of life
   • 7.8Average change in specific aspects of quality of life

8. Social/General functioning

   • 8.1Average endpoint general functioning score
   • 8.2Average change in general functioning scores
   • 8.3No clinically important change in specific aspects of functioning, such as social or life skills
   • 8.4Not any change in specific aspects of functioning, such as social or life skills
   • 8.5Average endpoint specific aspects of functioning, such as social or life skills
   • 8.6Average change in specific aspects of functioning, such as social or life skills

9. Adverse effects

   • 9.1Not any general adverse effects
   • 9.2Average endpoint general adverse effect score
   • 9.3Average change in general adverse effect scores
   • 9.4No clinically important change in specific adverse effects
   • 9.5Not any change in specific adverse effects
   • 9.6Average endpoint specific adverse effects
   • 9.7Average change in specific adverse effects
   • 9.8Death - natural and suicide

10. Economic Costs (excluding housing costs)

    • 10.1Direct costs
    • 10.2Indirect costs

Search methods for identification of studies

Electronic searches

The Cochrane Schizophrenia Group Trials Register (July 2009) will be searched using the phrase: [(*employ* or ((*supp* or *transitional*) and (*employ* or *work*) or ((*psychosocial* or *psycho-social* or *psychiatric* or *occupational or *soc* or *work* or *job* or *counsel*) and *rehab*) or *sheltered work* or *vocatio* or *fountain house* or *fountain-house* or *clubhouse* or *club-house* or *occupat* or *job* or *work therap* or *delivery of health care* or *delivery of integrated delivery* in title, abstract and index fields in REFERENCE) or (*vocat* or work* or *employ* or * job* or *occupat* or * placem* or *rehab*) in STUDY interventions)]

This register is compiled by systematic searches of major databases, hand searches and conference proceedings (see Group Module).

Searching other resources

1. Reference searching

   The sensitivity of the search strategy will be examined by comparing the results of the search with the reference lists of the identified reviews and trials to determine how many cited trials had not been detected.

2. Personal contact

   Researchers in the field will be approached to identify unpublished studies.

Data collection and analysis

Selection of studies

Two reviewers (YK and IMO) independently inspect all the citations identified by the search and request all potentially relevant articles, contacting the authors where necessary. Once the full articles have obtained, the two independent reviewers will decide whether the studies described therein meet the inclusion criteria. In the event of a disagreement, they will involve the third reviewer who will make a final decision. If it is not possible to obtain sufficient information to judge whether a study meets inclusion criteria, it will be placed in the list of studies awaiting assessment until such information becomes available.

Data extraction and management

1. Extraction

   Two independent reviewers (YK and IMO) will extract data from the selected trials using the double entry method. In the event of a difference between the reviewers, they will seek to resolve the difference by further scrutiny of the original trial reports, and may involve a third reviewer and/or contact the authors for further information.

2. Management

   Data will be extracted onto standard, simple forms.

3. Scale-derived data

   We will include continuous data from rating scales only if: (a) the psychometric properties of the measuring instrument have been described in a peer-reviewed journal (Marshall 2000); (b) the measuring instrument was not written or modified by one of the trialist; (c) the measuring instrument is either (i) a self-report or (ii) completed by an independent rater or relative (not the therapist).

Assessment of risk of bias in included studies

Two review authors (YK and IMO) will independently assess risk of bias in accordance with the Cochrane Collaboration’s tools for assessing quality and risk of bias (Higgins 2008). This tool encourages consideration of how the sequence was generated, how allocation was concealed, the integrity of blinding, the completeness of outcome data, selective reporting and other biases.

The risk of bias in each domain and overall are assessed and categorised into:

1. Low risk of bias: plausible bias unlikely to seriously alter the results (categorised as ‘Yes’ in Risk of Bias table)

2. High risk of bias: plausible bias that seriously weakens confidence in the results (categorised as ‘No’ in Risk of Bias table)

3. Unclear risk of bias: plausible bias that raises some doubt about the results (categorised as ‘Unclear’ in Risk of Bias table)

Trials with high risk of bias (defined as at least 3 out of 5 domains were categorised as ‘No’) will not be included in the meta-analysis. If the raters disagree, the final rating will be made by consensus with the involvement of another member of the review group. Where inadequate details of randomisation and other characteristics of trials are provided, authors of the studies will be contacted in order to obtain further information. Non-concurrence in quality assessment will be reported.

Measures of treatment effect

1. Binary outcomes

   Where binary outcomes (proportions) are used, we will calculate fixed-effects model relative risks (RR) (Furukawa 2002), and 95% confidence intervals for each outcome. In the event of significant heterogeneity, we will use a random effects model. The relative risk will be chosen over the odds ratio because the latter tends to overstate effect size when event rates are high (Higgins 2008).

2. Continuous data

   • 2.1Summary statistic

     For continuous outcomes we will estimate a Weighted Mean Difference (WMD) between groups. WMDs are based on the random-effects model as this takes into account any differences between studies even if there is no statistically significant heterogeneity. We will not calculate Standardised Mean Differences measures.

   • 2.2Endpoint versus change data

     Since there is no principal statistical reason why endpoint and change data should measure different effects (Higgins 2008), we will use scale endpoint data which is easier to interpret from clinical point of view. If endpoint data are not available, we will use change data.

   • 2.3Skewed data

     Continuous data on clinical and social outcomes are often not normally distributed. To avoid the pitfall of applying parametric tests to non-parametric data, we aim to apply the following standards to all data before inclusion: (a) standard deviations and means are reported in the paper or obtainable from the authors; (b) when a scale starts from the finite number zero, the standard deviation, when multiplied by two, is less than the mean (as otherwise the mean is unlikely to be an appropriate measure of the centre of the distribution, (Altman 1996); (c) if a scale starts from a positive value (such as PANSS which can have values from 30 to 210) the calculation described above will be modified to take the scale starting point into account. In these cases skew is present if 2SD>(S-S min), where S is the mean score and S min is the minimum score. Endpoint scores on scales often have a finite start and end point and these rules can be applied. When continuous data are presented on a scale which includes a possibility of negative values (such as change data), it is difficult to tell whether data are skewed or not. Skewed data from studies of less than 200 participants will be entered in additional tables rather than into an analysis. Skewed data pose less of a problem when looking at means if the sample size is large and they will be entered into syntheses.

   • 2.4Data synthesis

     When standard errors instead of standard deviations are presented, the former will be converted to the standard deviations. If standard deviations are not reported and can not be calculated from available data, authors will be asked to supply the data. In the absence of data from authors, the mean standard deviation from other studies will be used.

Unit of analysis issues

1. Cluster trials

   Studies increasingly employ ‘cluster randomisation’ (such as randomisation by clinician or practice) but analysis and pooling of clustered data poses problems. Firstly, authors often fail to account for intra class correlation in clustered studies, leading to a ‘unit of analysis’ error (Divine 1992) whereby p values are spuriously low, confidence intervals unduly narrow and statistical significance overestimated. This causes type I errors (Bland 1997; Gulliford 1999).

   Where clustering has not been accounted for in primary studies, we will present the data in a table, with a (*) symbol to indicate the presence of a probable unit of analysis error. In subsequent versions of this review we will seek to contact first authors of studies to obtain intra class correlation coefficients of their clustered data and to adjust for this by using accepted methods (Gulliford 1999). Where clustering has been incorporated into the analysis of primary studies, we will also present these data as if from a non-cluster randomised study, but adjusted for the clustering effect. The binary data as presented in a report should be divided by a ‘design effect’ (Raj 2005). This is calculated using the mean number of participants per cluster (m) and the intra-class correlation coefficient (ICC) [Design effect = 1+(m−1)*ICC] (Donner 2002). If the ICC has not been reported it will be assumed to be 0.1 (Ukoumunne 1999).

2. Studies with multiple treatment groups

   Where a study involved more than two treatment groups, if relevant, the additional treatment groups will be presented in additional relevant comparisons. Data will not not double counted. Where the additional treatment groups were not relevant, these data will not be reproduced.

Dealing with missing data

1. Overall loss of credibility

   At some degree of loss of follow up data must lose credibility (Xia 2007). Should more than 40% of data be unaccounted for we will not reproduce these data or use them within analyses.

2. Binary

   In the case where attrition for a binary outcome is between 0 and 40% and outcomes of these people are described, we will include these data as reported. Where these data are not clearly described, data will be presented on a ‘once-randomised-always-analyse’ basis, assuming an intention to treat analysis. Those lost to follow up will be all assumed to have a negative outcome. For example, for the outcome of employment, those who were lost to follow up all unemployed. A final sensitivity analysis will be undertaken testing how prone the primary outcomes are to change when ‘completed’ data only are compared to the intention to treat analysis using the negative assumption.

3. Continuous

   In the case where attrition for a continuous outcome is between 0 and 40% and completer-only data are reported, we will reproduce these.

4. Intention-to-treat (ITT)

   Intention-to-treat (ITT) will be used when available. We anticipate that in some studies, in order to do an ITT analysis, the method of last observation carried forward (LOCF) would be employed within the study report. As with all methods of imputation to deal with missing data, LOCF introduces uncertainty about the reliability of the results. Therefore, where LOCF data have been used in the analysis, it will be indicated in the review.

Assessment of heterogeneity

1. Clinical heterogeneity

   We will consider all included studies, hoping to use all studies together. Should clear unforeseen issues be apparent that may add obvious clinical heterogeneity, we will note these issues, consider them in analyses and undertake sensitivity analyses for the primary outcome.

2. Statistical

   • 2.1Visual inspection

     We will visually inspect graphs to investigate the possibility of statistical heterogeneity.

   • 2.2Employing the I-squared statistic

     Heterogeneity between studies will be investigated by using the I-squared method (Higgins 2003) and the Chi-squared ‘p’ value. The former provides an estimate of the percentage of variation in observed results thought unlikely to be due to chance. A value equal to or greater than 50% will be taken to indicate heterogeneity and reason for heterogeneity will be explored. If the inconsistency is high and the clear reasons were found, the data will be presented separately.

Assessment of reporting biases

Data from all identified and selected trials will be entered into a funnel graph (trial effect versus trial size) in an attempt to investigate overt publication bias. The possible existence of small study effects will be examined by Egger’s regression method (Egger 1997) as well as by visual inspection of the graph.

Data synthesis

In the absence of significant heterogeneity, a fixed-effect model will be used. However, if significant heterogeneity is demonstrated, a random-effects model will be used for analysis. Where available, the analyses will be based on intention-to-treat data from the individual studies. The data from included trials will be combined in a meta-analysis if they are sufficiently homogeneous, both clinically and statistically.

Subgroup analysis and investigation of heterogeneity

1. Pre-planned subgroup analyses

   Subgroup analyses should be performed and interpreted with caution because multiple analyses will lead to false positive conclusions (Oxman 1992). However, we will perform the following subgroup analyses, where possible, for the following a priori reasons:

   1. Excluding studies with low fidelity IPS and augmented supported employment.

   2. Augmented supported employment versus treatment as usual

2. Regression analyses

   If we have a sufficient number of trials (roughly 9 -11) per independent variable, meta-regression will be performed to determine whether various study-level characteristics affect effect sizes. Possible effect modifiers to be examined include: study location (USA versus other countries), study location (urban versus rural) and the local unemployment rate. STATA will be used to perform the meta-regression (STATA 2005).

Sensitivity analysis

We will examine the robustness of our findings by excluding (i) studies with less than 20% follow up on the variable at the time point (ii) skewed data (iii) trials with a high risk of bias or where the overall risk of bias was unclear, and (iv) studies where IPS was augmented with other interventions.

WHAT’S NEW

Date	Event	Description
6 October 2010	Amended	Contact details updated.

HISTORY

Protocol first published: Issue 1, 2010

Date	Event	Description
15 February 2010	Amended	Contact details updated.