Randomized controlled trial of supported employment in England: 2 year follow-up of the Supported Work and Needs (SWAN) study

Abstract

Studies from North America have concluded that supported employment using the Individual Placement and Support (IPS) model is effective in helping individuals with severe and persistent mental illness gain competitive employment. The aim of this study was to investigate the effectiveness and cost-effectiveness of IPS in England in patients followed up for 2 years. Patients with severe mental illness were randomised to IPS or local vocational services (treatment as usual). Service use and costs were measured. Two hundred-nineteen participants were randomised, and 86% re-assessed 2 years later. In the multivariate analysis, relatively low rates of competitive employment were found in both the intervention group and the treatment as usual group, although significantly more patients obtained competitive employment in the treatment arm (22% vs. 11%, p=0.041). There were no significant differences in costs. The employment rate among participants receiving IPS was lower than in previously published reports, and the number needed to treat to obtain the benefit of IPS was relatively high. This may reflect difficulties in the implementation of IPS where it is not structurally integrated within mental health teams, as well as economic disincentives which lead to lower levels of motivation for patients and mental health professionals.

The Individual Placement and Support (IPS) model of supported employment was developed in the 1990s and is focussed on vocational rehabilitation as part of mental health treatment rather than a separate entity 1.

IPS aims for rapid job placement into competitive employment (i.e., in the open labour market) followed by support and necessary training obtained while in the job. IPS services are integrated within community mental health services and based on clients preferences 1.

Randomized controlled trials (RCTs) in the United States have found this model of vocational rehabilitation to be more effective in gaining employment for people with severe mental illness when compared with traditional vocational rehabilitation models 2,3,4,5,6,7,8,9. This has also been replicated outside North America 10,11. However, until the SWAN (Supported Work and Needs) trial 12, an RCT of supported employment in South London, there had been no large trials of supported employment in the UK.

The SWAN study reported that at one year follow-up the rate of employment was low for both the intervention group (13%) and the control group (7%), with no significant difference between the groups 12. Our primary hypothesis was that at 2 years follow-up a significantly greater percentage of individuals who received IPS would be in competitive employment compared with those receiving usual services. We report here on the 2 year follow-up findings.

METHODS

The SWAN (Supported Work and Needs) study is a pragmatic RCT of the IPS model of employment. Participants were recruited from community mental health teams in two boroughs of South London. Inclusion criteria were that participants should be receiving outpatient or community psychiatric care from local mental health services, have severe mental illness (duration of illness over 2 years, Global Assessment of Functioning score of 60 or less, and a diagnosis of a psychotic or chronic affective disorder), be aged 18-65, be able to read and speak English to a high enough standard to give informed written consent, and have been unemployed for at least 3 months. After researchers had given a full description of the study to the potential participants, written informed consent was obtained. Ethical approval was obtained from the Joint Institute of Psychiatry/South London and Maudsley Research Ethics Committee.

Participants were randomly allocated to the intervention or the control group. Treatment allocation was stratified by gender and age (10 year bands). Randomization with minimization was used, performed by the Institute of Psychiatry Mental Health and Neuroscience Clinical Trials Unit, a unit independent of the study to maintain concealment. The intervention condition was an IPS programme integrated within community mental health teams, provided by a well-established not-for-profit non-governmental supported employment agency. The intervention involved linking the four employment specialists (two for each local borough) with community mental health teams, focussing on rapid placement with continued follow-up support. The employment specialists also sought to find employment opportunities that were consistent with participants’ preferences, skills and abilities. The control condition (treatment as usual, TAU) consisted of existing psychosocial rehabilitation and day care programmes available in the local area 12.

Participants were assessed at baseline by the recruiting researcher and at 12 months and 24 months after randomization by a different researcher who was blind to allocation status. Each participant was given £20 for their time at the baseline and follow-up interviews. Participants were not reimbursed for any contacts they had with the employment consultants.

At baseline, participants were assessed using the Structured Clinical Assessment in Neuropsychiatry (SCAN, 13) to determine their diagnosis. Participants with a SCAN diagnosis of schizophrenia, schizoaffective disorder, delusional disorder or other psychotic disorders were categorized as having a psychotic disorder. Participants with a SCAN diagnosis of bipolar disorder, mania or depression were categorised as having a mood disorder.

At the baseline, 1-year and 2-year interviews, data was collected on demographic information, service use and employment status over the previous 12 months. Measures of psychosocial functioning were: the Manchester Short Assessment (MANSA) version 2 14, a brief modified version of the Lancashire Quality of Life Profile, scored from 1-7 with low scores representing a lower quality of life; the Camberwell Assessment of Need (CAN) short version 15, an interview measure for assessing the unmet needs of people with severe mental illness in 22 health and social domains; the Rosenberg Self-Esteem Scale (RSE, 16), which is scored from 1-4 with low scores representing lower self esteem; the Brief Psychiatric Rating Scale (BPRS, 17), a researcher rated measure of psychopathology with possible scores ranging from 24 to 168, low scores representing fewer symptom); the Global Assessment of Functioning (GAF, 18), a researcher rated measure of functioning with a range of 0 to 100, low scores representing lower levels of functioning; and the Client Service Receipt Inventory (CSSRI, 19), a measure of health and social resource use during the previous 12 months from patient report.

Open competitive employment was defined as a job paying at least the minimum wage, located in a mainstream socially integrated setting not set aside for persons with disabilities, held independently (i.e., not agency owned), with the participant in continuous employment for at least 30 days (with part-time employment rated pro-rata).

We calculated that a sample size of 75 in each group (150 in total) would be sufficient to detect a difference in employment from 10% in the TAU group compared to 30% in the experimental group, assuming a significance level p=0.05 (double sided) at 80% power. Allowing for a 30% attrition rate, our target recruitment was 108 (216 in total) in each group. The base rate of 10% and the likely refusal and attrition rates were based on the study in Baltimore 7 and are typical of many populations of people with schizophrenia in England.

Due to the nature of the study, it was not possible for participants or those administering the intervention to be blind to the participants’ allocation status. However, the researcher who conducted the 1 and 2 year follow-up interviews was blind to allocation status. When conducting the follow-up assessment, the researcher was not told the participant’s allocation status and asked the participant not to disclose the allocation if possible. A test of blindness was carried out by comparing the researcher’s best guess allocation with the actual allocation.

The IPS fidelity scale 20 was completed by a lead occupational therapist independent from the study. During the course of the study, employment workers were required to record any events that occurred in relation to the client. These events included direct client contact and any non-client contacts that were regarding the client, e.g., contact with the client’s community psychiatric nurse. Employment workers were required to record the amount of time spent per contact, where and when the contact took place and the nature of the contact.

All data were analysed using SPSS for Windows (version 15.0). The primary and secondary hypotheses were tested on the whole group. We used t and c2 tests to compare means and proportions respectively, unless the data were highly skewed, in which case non-parametric tests were used. Logistic and linear regression models were also fitted including potential confounding variables or variables associated with missing status: (grouped) age, gender, ethnic group, educational level, symptomatology, functioning and diagnosis. All data were analysed in groups as randomized, whether or not receiving an intervention (i.e., intention to treat). Data were compared for those followed up with those not followed up, overall and by each treatment arm.

Service use data measured with the CSSRI were combined with appropriate unit cost information 21. Medication for psychosis, mania, depression, anxiety were costed, along with medication to treat side effects of these medications, using prices from the March 2006 British National Formulary 22. If any client reported using a medication but did not know the dose, the lowest recommended dose was used to cost the drug. Cost data are typically skewed and therefore bootstrap methods were used to produce a confidence interval around the mean cost difference between the groups. Cost data were combined with the main outcome (proportion of participants in employment during the two year follow-up period). If costs were lower for IPS and outcomes better then it would be “dominant”. Higher costs and better outcomes would require the use of an incremental cost-effectiveness ratio to show the extra costs incurred to achieve a one percentage point difference in participants employed. A cost-effectiveness acceptability curve was produced to show the probability that IPS was more cost-effective than standard care for a range of values placed on a unit improvement in outcome. The range used here was £0-1000 in £100 increments. There were no data to guide the choice of this range, but it was assumed that gains in employment would be valued in terms of £100s.

RESULTS

Participants were recruited between November 2004 and September 2006. Of the 375 people referred to the study, 220 entered into the study (31 did not meet inclusion criteria; 108 refused to participate; 17 were excluded for other reasons). One participant withdrew between the baseline assessments and randomization, leaving 219 participants (Figure 1). There were no substantial differences between the two randomized arms on any baseline variables (see Table 1).

Figure 1 CONSORT flow diagram.

Table 1.

Table 1 Baseline socio-demographic and clinical variables of participants by randomization group

	Control	Intervention
Males (%)	66	69
Borough A (%)	43	35
Borough B (%)	57	65
Worked in the last 5 years (%)	56	51
Living alone (%)	53	55
Ethnicity (%)
White	41	34
Black	41	45
Other	17	20
Diagnosis (%)
Psychotic disorder	69	76
Mood disorder	31	24
Age (mean ± SD)	38.3±9.3	38.4±9.5
Rosenberg Self Esteem Scale score (mean ± SD)	2.3±0.5	2.2±0.6
Global Assessment of Functioning score (mean ± SD)	47.7±6.3	48.3±7.0
Overall Manchester Short Assessment score (mean ± SD)	3.9±0.7	4.0±0.7
Brief Psychiatric Rating Scale score (median, interquartile range)	33 (28-41)	33 (28-42)
Camberwell Assessment of Need unmet needs (median, interquartile range)	2 (0-3)	1 (0-3)

Thirty participants were lost to follow-up at 2 years, with similar proportions in both intervention and control groups. There were no significant baseline differences in socio-demographic or clinical variables between those who were and those who were not lost to follow-up; however, the proportion having worked in the last 5 years was somewhat higher among those lost to follow-up, albeit not significantly (66% vs. 52%, χ²₁=1.738, p=0.19).

IPS fidelity was found to be high; the employment consultants in the 2 boroughs maintained a good IPS rating of 69 and 67 respectively. For participants who had contact with an employment consultant (93 participants, 85%), the mean number of contacts with or on behalf of the clients was 15.41 (SD 17.46, median 9, range 1-77). Sixteen (15%) patients made no contact with the employment consultant despite being offered appointments to do so. One participant in the control arm had contact with the intervention.

The researchers who conducted the follow-up interviews guessed the allocation status correctly in 220 out of the 386 (57%) interviews at 1 year and 2 year follow-up, compared to a hypothesized 50% with random guesses (p=0.006; single sample, double sided test of a proportion).

Thirty-two (17%) participants out of 190 who were followed up reported having worked, between baseline and the 2 year follow-up, in jobs that met the competitive employment criteria. Of those followed up, 11 out of 95 (11%) were from the control group and 21 out of 95 (22%) were from the intervention group (risk ratio 1.91; 95% CI 0.98 to 3.74; χ²₁=3.758, p=0.053). This is equivalent to a number needed to treat (NNT) of 9. The 30 participants who were not available for follow-up were counted as missing for this analysis.

A logistic regression analysis was conducted controlling for all socio-demographic factors and clinical measures at baseline. This revealed that whether the participant had a job in the last 5 years before baseline also predicted outcome (p=0.001), and that the effect of the intervention was statistically significant (p=0.041). Sensitivity analyses found that when assuming all participants who were not followed up had not worked, the results were also significantly in favour of the intervention (χ²₁=3.768, p=0.052), though this was not the case when assuming all participants who were not followed up had worked (χ²₁=1.956, p=0.162). Sensitivity analyses also found that the p values for arm when controlling for potential confounders in the analysis adjusted using logistic regression were p=0.044, assuming all participants who were followed up had not worked, and p=0.074, assuming all participants who were followed up had worked.

Overall, the whole group was followed up for a mean of 703 days (SD 95.5). For the intervention arm this was 708 days (SD 87.1) and for the control 698 days (SD 103.1) (t₂₀₂=0.723, p=0.470). A Kaplan-Meier survival analysis found that the estimated mean time to getting a job was 680 days (95% CI 648-712) for the control group and 630 days (95% CI 588-671) for the intervention group. The time to obtaining a job was thus slightly lower by an average of about 6 weeks for the intervention arm (χ²₁=3.753, p=0.053). In a Cox regression adjusting for any significant potential confounders (having a job in the last 5 years, CAN score and GAF score), arm was significant in estimating length of time until finding a job (p=0.043).

Of the 32 people who had a job during the 2 year follow-up period, 25 participants had one job, 6 participants had two different jobs and one participant had three different jobs. There was a trend for a higher median number of hours worked per week for the control group compared to the intervention group (36 vs. 15 hours; Wilcoxon Z = -1.765, p=0.078). However, there was no evidence for a difference in duration of jobs between the 11 people who obtained jobs in the control group and the 21 people in the intervention group (median 13 vs. 22 weeks; Z=-0.955, p=0.353). There was a significant difference in the salary per hour between the two groups, with the control group tending to earn higher salaries than the intervention group (median £8.81 vs. 6.00 per hour; Z=-2.146, p=0.031).

The majority of jobs gained in the 2 year follow-up period were elementary occupations or sales and customer service occupations (14/40, 35% and 11/40, 27.5% respectively). There were 6/40 (15%) administrative and secretarial occupations, 2/40 (5%) process, plant and machinery operatives, 2/40 (5%) skilled trade occupations and 2/40 (5%) professional occupations. There was also one manager/senior official and one associated professional/technical occupation (both in the control arm) and one personal service occupation (combined as 7.5%). These jobs were classified using the Standard Occupational Classification 23.

There was no significant difference in employment outcome by borough: within borough A, 9 (12.5%) of participants obtained competitive employment whereas within borough B, 23 (19.5%) of participants obtained competitive employment (χ²₁=1.561, p=0.212).

There were no differences between the intervention and the control groups at follow-up on any of the clinical measures (see Table 2).

Table 2.

Table 2 Clinical outcome for participants by randomization status

	Control	Intervention	t, Z	p
Camberwell Assessment of Needs unmet needs (median, interquartile range) (n=183)	2 (0-4.7)	1 (0-3.25)	Z= -1.574	0.12
Brief Psychiatric Rating Scale score (median, interquartile range) (n=185)	33.5 (28-44.5)	35 (29-43)	Z= -0.225	0.82
Rosenberg Self Esteem Scale score (mean ± SD) (n=183)	22.9±6.1	22.3±5.3	t181= 0.731	0.47
Overall Manchester Short Assessment score (mean ± SD) (n=178)	3.9±1.1	4.1±0.9	t176=-1.445	0.15
Global Assessment of Functioning score (mean ± SD) (n=189)	56.8±18.7	56.1±16.4	t187= 0.300	0.77

At the end of the 2 year follow-up period, there were adverse events for four participants. Two participants were reported as missing by family and care coordinators (one from the intervention group and one from the control group). One participant was deceased as a result of a suspected medication overdose (control group). One participant was charged with murder (intervention group). None of these events were related to the study or the intervention.

There was a high level of use of general practitioners, psychiatrists and community mental health nurses in both arms. Inpatient costs were higher than for other services. The only service for which there was a noticeable cost difference between the groups at 2 year follow-up was day care and education, where costs were higher for the control group. The average intervention costs were slightly below £300. There were no significant differences between the intervention and control arm on overall costs, service costs or medication costs over the 2 year period (see Table 3). Regression analysis showed a cost difference of £2361 in favour of the intervention, but this was not statistically significant (-£6105 to £1308).

Table 3.

Table 3 Service use and costs over the entire 2 year follow-up period

	Control			Intervention
	Using service	Contacts	Cost	Usingservice	Contacts	Cost
	(%)	(mean±SD)	(mean±SD)	(%)	(mean±SD)	(mean±SD)
Psychiatric inpatient	23	19.9±59.8	4173±12340	20	17.4±55.8	3168±10754
Psychiatrist	66	2.0±3.7	1886±3121	56	1.5±2.2	1335±1896
Other doctor	30	0.8±2.8	239±770	37	0.9±1.7	227±538
Day care/education	30	11.5±26.1	2102±5878	17	4.6±12.5	562±1515
Psychologist	8	0.4±1.8	131±623	13	0.9±3.8	370±1675
Social care	16	1.2±4.4	209±726	12	0.6±2.4	111±404
General inpatient	13	2.1±10.2	469±2408	13	1.1±5.4	253±1268
General practitioner	77	2.3±2.4	411±524	70	2.7±4.3	430±569
District nurse	8	0.3±1.1	9±45	7	0.3±1.1	11±45
Community mental health nurse	81	7.5±7.3	839±1271	85	7.4±8.6	761±947
Occupational therapist	1	0.0±0.4	3±27	2	0.1±0.9	17±113
Other service	6	0.2±0.8	16±77	11	1.4±7.1	240±1307
Medication	97		1767±1830	98		1841±1807
Total costs			11932±13694			9571±11217
Inpatient data were collected for the entire 24 month period. Other service use data were collected for the 3 month period prior to 12- and 24-month follow-up interview. Service contacts for inpatient care are the number of days. Costs are over a 24 month period and are in 2006/7 £s (non-inpatient data was multiplied by 4 to get yearly figures for the 1 and 2 year follow-up)

Based on the point estimates of costs and outcomes, IPS was seen as dominant. The cost-effectiveness acceptability curve showed that, even if a value of £0 is placed on one more person gaining employment, there is still a 90% likelihood that IPS is the most cost-effective option.

DISCUSSION

In this study we found that IPS was significantly more effective in helping patients with severe mental illness to obtain competitive employment compared with traditional vocational services at 2 year follow-up, but that a low proportion of patients in both groups obtained employment, with even the intervention group having lower rates than reported in control groups in previous RCTs. Similarly, time to achieve employment was shorter in patients in the intervention arm (by six weeks), but this was with a long delay of more than one and a half years from randomization.

Of the seven previous RCTs that reported time to first job 2,4,5,6,7,8,9,11, all found that the typical time to first job was longer for the control than the intervention group, but time to first job in the intervention group varied from 72 to 197 days, whereas in the control group this was 118-293 days. In addition, the number needed to treat (NNT=9) reflects the fact that, while the proportion gaining a job in the intervention arm is double the proportion in the control arm, the absolute levels, and hence absolute difference, are relatively modest.

Therefore, at both 1 year 12 and 2 year follow-up, IPS was less effective than previously reported. The IPS arm had lower costs than the control arm, but this difference was not statistically significant. While the point estimates of costs and outcomes suggest that IPS may be cost-effective, this needs to be viewed alongside the fact that the difference in outcomes was not substantial.

These findings may be due to differences in the England labour market compared to that in North America, namely a lack of employer incentives in employing patients with mental illness and differences in the benefits system. There was also a relatively high proportion of participants from non-white ethnic groups in this study. This could limit the success of IPS in England, as people who are not white are more likely to be unemployed in England 24.

In some of the RCTs in the USA, participants had to attend two initial meetings to screen out those who were less motivated, whereas in our study there was no such screening, reflecting current UK practice. The study may therefore have demonstrated greater effectiveness of IPS if participants were screened for motivation before being permitted into the study, or had received some form of motivational interviewing prior to entering the IPS programme. Also of note, this study was conducted in deprived areas of London, with a higher rate of unemployment compared to the national average, where it may be difficult to achieve even moderate rates of competitive employment.

Finally, the IPS program was provided by an external support employment agency. Such well-established but financially insecure non-governmental agencies may not be as IPS focused as new employment agencies set up as IPS programs (as in previous RCTs). Despite the high IPS fidelity rating, the separation of the supported employment agency from mental health services may also have led to differences in the intervention received by patients in this study compared with that delivered in more fully integrated models of care.

There is some evidence that our study population was more disabled than that recruited in other studies. This group may need longer periods of support and this suggests that time-limited IPS may not be as helpful for this particularly ill group.

This study is the largest RCT of IPS in a non-US setting, with a high level of follow-up and a diverse group of participants. With regard to the likely effect of drop-out, sensitivity analyses only cast doubt on the findings in the extreme scenario that all dropouts found work (which is unlikely). For the finding on proportions worked, adjustment for the only variable possibly related to loss to follow-up (having worked in the last 5 years) made little difference to the conclusions. While there was evidence of unblinding in a small proportion of cases, the primary outcome was an objective measure of finding a job, so this should have little impact on our results.

A key strength of this study is that it was conducted in a “real world” setting in a socially deprived inner-city catchment area where most patients with a severe mental illness do not obtain employment. The findings from this study are also important in highlighting the difficulties that are faced when implementing the IPS model in a non-US setting with the most severely mentally ill patients.

We conclude that, although IPS has produced beneficial results, these are rather more modest in this study than in previous research, and additional interventions may need to be provided to promote social inclusion for the majority of people with severe mental illness.