Who Benefits From Supported Employment: A Meta-analytic Study

Abstract

Aims: This meta-analysis sought to identify which subgroups of clients with severe mental illness (SMI) benefited from evidence-based supported employment. Methods: We used meta-analysis to pool the samples from 4 randomized controlled trials comparing the Individual Placement and Support (IPS) model of supported employment to well-regarded vocational approaches using stepwise models and brokered services. Meta-analysis was used to determine the magnitude of effects for IPS/control group differences within specific client subgroups (defined by 2 work history, 7 sociodemographic, and 8 clinical variables) on 3 competitive employment outcomes (obtaining a job, total weeks worked, and job tenure). Results: The findings strongly favored IPS, with large effect sizes across all outcomes: 0.96 for job acquisition, 0.79 for total weeks worked, and 0.74 for job tenure. Overall, 90 (77%) of the 117 effect sizes calculated for the 39 subgroups exceeded 0.70, and all 117 favored IPS. Conclusions: IPS produces better competitive employment outcomes for persons with SMI than alternative vocational programs regardless of background demographic, clinical, and employment characteristics.

Introduction

Supported employment has been widely recognized as an evidence-based practice for individuals with severe mental illness (SMI).¹ The most clearly described and widely researched supported employment model is Individual Placement and Support (IPS),² which is based on 7 principles: (a) focus on competitive employment outcomes, (b) open to anyone who wants to work, (c) rapid job search, (d) attention to client preferences in services and job searches, (e) individualized and long-term supports, (f) employment specialists work closely with treatment team, and (g) clients receive personalized counseling on Social Security and other benefits. The essence of IPS is a direct, individualized search for competitive employment that eschews prolonged preemployment preparation, counseling, and training.

Five reviews^3–7 and 1 multisite randomized trial⁸ have examined the effectiveness of supported employment across multiple randomized controlled trials (RCTs). All concluded that supported employment was more effective in helping clients with SMI obtain competitive employment than were alternative approaches, with overall effect sizes for experimental differences in competitive employment rates ranging from 0.43 to 0.79. The most recent of these reviews focused specifically on RCTs evaluating high-fidelity IPS programs compared with a variety of alternative vocational approaches.⁴ It included 11 RCTs conducted in a variety of geographic regions and client populations, each showing significantly better competitive employment outcomes. Based on examination of a variety of indicators of competitive employment outcomes, IPS consistently had better outcomes than traditional vocational approaches. Specifically, this review found that the 61% of IPS participants obtained a competitive job during follow-up, compared with 23% of controls. About two-thirds of IPS participants who obtained competitive employment worked 20 hours or more per week. Among those who obtained a competitive job, time to first job averaged 138 days for IPS participants compared with 206 days for controls, a difference of 10 weeks.

One common perception of supported employment is that participants have limited job tenure. The best evidence on this question comes from long-term follow-up studies. Two long-term follow-up studies suggest that over half of those who enrolled in IPS have become steady workers, employed at least 50% of the time over a 10-year period after enrollment in IPS.^9,10 Two other long-term studies also suggest encouraging long-term employment outcomes for those who enter supported employment programs.^11,12

Despite generally positive outcomes for IPS, observers have noted limitations as well.¹³ For example, earnings from competitive employment for supported employment participants are typically quite modest. One multisite study reported monthly employment earnings of $122 for supported employment participants, which was significantly higher than $92 per month for controls.⁸ The amount of employment income supported employment participants receive is strongly influenced by the disincentives associated with receiving Social Security disability payments.¹⁴

While the clear advantage of IPS over traditional vocational approaches has been widely replicated in general samples of clients with SMI, the generalizability of the IPS model across different subgroups has not been studied extensively. While there are a handful of studies of specific target groups, such as Social Security Disability Insurance (SSDI) beneficiaries,¹⁵ young adults recently diagnosed with schizophrenia,¹⁶ and middle-aged and older adults,¹⁷ the effectiveness of supported employment for many subgroups remains undemonstrated, and critics continue to argue that some clients lack readiness and therefore need some form of prevocational training.^18–20 Significantly, one of the core principles of supported employment has always been “zero exclusion,” ie, that no one who expresses a desire to work is excluded from receiving supported employment on the basis of personal characteristics.²¹ This principle expresses an important value, namely, that supported employment services should be accessible to all people with disabilities, irrespective of level of disability, symptomatology, work history, or other factors assumed (or shown) to be predictors of poorer outcomes. Beyond this value statement, however, is an empirical question: Are there some types of clients who would be better served by sheltered employment, transitional employment, or another prevocational training model? Such assertions are commonly made, eg, with regard to individuals with poor work histories or severe psychiatric symptoms.

Historically, many vocational approaches have assumed that some clients are not ready for employment because of their vocational deficits.¹⁹ In fact, many widely held assumptions about factors predicting poor vocational outcomes have not proven to be correct. For example, presence of co-occurring substance use disorders is not associated with poorer employment outcomes.²² Prediction of vocational outcomes from client characteristics has proved to be difficult, with reviews yielding disparate conclusions.²³ An important qualification noted in several reviews is that client factors have been much stronger predictors of vocational outcome in community samples of individuals with SMI than in supported employment samples.²⁴ These reviews further suggest that supported employment services, especially those augmented with cognitive interventions,^25–27 can compensate for cognitive impairments and deficits that would otherwise result in poorer employment outcomes.

Individual RCTs of supported employment often have not had sufficient power to examine individual client subgroups, although one multinational RCT did have a larger sample size and hence a greater opportunity to examine client subgroups.²⁸ The drawback to this multinational study, however, was that sociopolitical and cultural factors may have overwhelmed any client factors. Another multisite project aggregating findings from 7 RCTs of supported employment had a large sample size,^29,30 but it was hampered by a lack of comparability among program models used, obfuscating the interpretation of the benefits of supported employment for different client groups.

The current article examines the question: Among various subgroups of clients with SMI (defined by work history, demographic, and clinical variables), which subgroups benefit from evidence-based supported employment? Alternatively, which subgroups benefit more from brokered stepwise vocational models?

Methods

Overview

This meta-analytic study used archival data from 4 independent RCTs to determine the magnitude of effects for IPS within specific client subgroups (defined by 2 work history, 7 sociodemographic, and 8 clinical variables) on 3 competitive employment outcomes (obtaining a job, total weeks worked, and job tenure).

The original projects were approved by institutional review boards at local sites and participating universities. In addition, the data reanalyses were approved by the Institutional Review Board of Indiana University-Purdue University Indianapolis.

Sample and Procedures

The sample consisted of study participants from 4 RCTs of IPS vs usual services.^31–34 All 4 studies compared a newly established IPS program with one or more well-established vocational programs. In all 4 studies, participants were recruited from mental health centers (or a psychiatric rehabilitation agency in the Chicago Study). Participants were adults who met each state's criteria for SMI, typically a Diagnostic and Statistical Manual of Mental Disorders (Fourth Edition) (DSM-IV) Axis I or II diagnosis plus severe and persistent impairment in psychosocial functioning. All participants were unemployed at the time of study admission. Other common eligibility criteria included desire for competitive work, ability and willingness to give informed consent, and absence of significant medical conditions that would preclude working or participating in assessment interviews. In addition, all participants were required to attend multiple research information meetings in which the project was explained, after which they gave written informed consent to participate in the study.³⁵ All 4 studies used a similar protocol to track employment outcomes and psychiatric symptoms. Importantly, none of the studies excluded clients on the basis of illness severity, lack of readiness, poor job history, or other clinical factors.

Data Sources

Despite similar methods, the RCTs differed on geographic location, control group interventions, and length of follow-up, as summarized in table 1. The 4 studies were as follows.

Table 1.

Comparison of 4 Randomized Controlled Trials of IPS

	Manchester and Concord, NH (Drake et al32)	Washington, DC (Drake et al33)	Hartford, CT (Mueser et al34)	Chicago, IL (Bond et al31)
IPS sample size	n = 73	n = 74	n = 68	n = 92
Comparison	Group skills training (n = 67)	Enhanced vocational rehabilitation (n = 76)	Psychosocial rehabilitation program (n = 67) and standard services (n = 69)	Diversified placement approach (n = 95)
Enrollment	January 1991–Jan 1994	January 1994–January 1997	April 1996–October 1998	September 1999–March 2002
Additional enrollment criteria	Attend research info groups, aged 20–65 y, clinical stability ≥1 mo, local residence ≥6 mo, unemployed ≥1 mo, no significant physical illness, memory impairment, or substance dependence	Attend research info groups, no significant physical illnesses	Attend research info groups, not competitively employed	Attend research info groups, aged ≥18 y, agency membership ≥30 d, not competitively employed during past 3 mo, no significant physical illness
Baseline characteristics of IPS sample	97% Caucasian, 49% male, aged 36.5 ± 9.8 y, 78.1% with ≥high school education, 42.5% with schizophrenia spectrum, 6.2% alcohol abuse, 1.6% drug abuse, 84.9% living independently, 49.3% never married	65% African American, 58% male, aged 38.2 y (SEM = 7 y), 67.1% with ≥high school education, 71.1% with schizophrenia spectrum, 13.2% alcohol abuse, 22.4% drug abuse, homeless days during past year: 14.2 ± 4.4 d, 68.4% never married	46% African American, 31% Hispanic, 62% male, aged 41.7 ± 8.8 y, 50.0% with ≥high school education, 76.5% with schizophrenia spectrum, 11.8% alcohol abuse, 22.4% drug abuse, 73.5% never married, competitive work in last 5 y: 42.6%	53% African American, 62% male, aged 39.5 ± 9.6 y, 81.2% with ≥high school education, 62.5% with schizophrenia spectrum, 78.1% never married, 26.1% living independently, paid work in last 5 y: 83.3%
Follow-up	18 mo	18 mo	24 mo	24 mo
Follow-up rate	97.9% @ 18 mo	94.7% @ 18 mo	81.4% @ 24 mo	88.1% @ 24 mo
Symptom scale	Expanded BPRS	Expanded BPRS	PANSS	PANSS

Note: IPS, Individual Placement and Support; BPRS, Brief Psychiatric Rating Scale; PANSS, Positive and Negative Syndrome Scale.

The “NH Study”³² was conducted in 2 mental health centers in Concord and Manchester, NH. It compared IPS with group skills training, in which a rehabilitation agency unaffiliated with the mental health centers provided prevocational skills training, individualized placement, and follow-along support.

The “DC Study”³³ recruited clients in an intensive case management program in Washington, DC. IPS was compared with enhanced vocational rehabilitation, in which rehabilitation agencies provided stepwise vocational services and primarily paid work adjustment training in a sheltered workshop.

In the “Hartford Study,”³⁴ clients receiving services at a mental health center in Hartford, CT, were randomly assigned to IPS, a psychosocial rehabilitation program, or standard services. Recruitment procedures ensured that one-third of the sample was Latino. Psychosocial rehabilitation program participants attended a series of preparatory training activities, followed by transitional employment and assistance with obtaining competitive work. Standard services involved access to 2 off-site employment programs, one providing brokered supported employment (which lacked the integrated services offered by IPS) and the other providing supervised janitorial enclaves.

In the “Chicago Study,”³¹ clients attending 2-day programs at a comprehensive psychiatric rehabilitation agency in Chicago, IL, were randomly assigned to IPS or a diversified placement approach, which offered stepwise preparation for competitive employment, including prevocational training, agency-run business opportunities, and group and individual placements. Both vocational programs were offered at 2 locations, Thresholds North and Thresholds South.

Sample Recruitment.

Among clients attending informational groups, the rates of consenting to participate were as follows: NH Study (50.4%),³⁵ DC Study (76.0%),³⁶ Hartford Study (72.1%),³⁴ and Chicago Study (67.6%).³¹ Overall, 699 (65.8%) of 1063 clients agreed to participate across the 4 studies. Reasons for nonparticipation were diverse and included the lack of a vocational goal and concerns about losing benefits. Six Chicago clients were administratively dropped shortly after study entry, leaving a final intent-to-treat sample of 693.

Missing Data.

Twelve participants (7 in IPS and 5 controls) with incomplete follow-up data were excluded from analyses. The authors of the Hartford Study included the 6 cases with incomplete vocational data in their intent-to-treat analyses, with zeros substituted for work during periods of missing data. Across the 4 RCTs, the combined sample size was 681 (IPS sample N = 307, comparison sample N = 374).

Standardization of IPS

The IPS program model was standardized across the 4 studies by ensuring fidelity to the IPS model through intensive training/consultation and ongoing monitoring during using the IPS Fidelity Scale.³⁷ This fidelity scale describes 15 critical features of the IPS model (eg, rapid job search, attention to client preferences) that are well described in the literature.²

Common Features of Programs Used in the Comparison Interventions

All the comparison interventions were highly regarded, active programs considered at the time to be state of the art. Common principles shared among these comparison groups were the emphasis on stepwise entry into competitive employment (with the exception of one subprogram in the Hartford Study) and brokered services in which the vocational program was provided by a separate agency from the mental health program (with the exception of the Chicago program).

Measures

Each predictor variable was used to define 2 or more mutually exclusive subgroups, as described below. For some variables (eg, gender, diagnosis), the choice of subgroups was obvious. For the symptom measures, the empirical literature was used to define subgroups based on symptom severity. For other variables (eg, amount of work history), a median split was used.

Work History and Sociodemographic Predictor Measures.

Demographics, psychiatric history, source of income, and work history were assessed at the baseline interview and by review of medical records. The variables and corresponding subgroups examined were as follows: paid employment in a community job in preceding 5 years (yes/no), weeks in paid work in last 5 years (median split of total sample at 17.3 wk), age (<45 y of age [n = 495], 45 y or older [n = 186]), gender (men, women), race/ethnicity (Caucasian, African American, or Latino), educational level (<high school, high school graduate, or >high school), marital status (never married, married/living together, divorced/separated/widowed), disability benefits (SSDI beneficiaries, supplemental security income [SSI] recipients, both SSI and SSDI, or neither), and homelessness in last year (yes/no). The cut point for age was based in part on the classification used in a number of studies of older and middle-aged adults.

Clinical Predictor Measures

Diagnosis.

Participants’ psychiatric diagnoses were determined at baseline using the Structured Clinical Interview for DSM (SCID), a semistructured diagnostic instrument. In the NH and DC studies, a version for Diagnostic and Statistical Manual of Mental Disorders (Third Edition Revised)³⁸ was used; the Hartford and Chicago studies used a version for DSM-IV.³⁹ Good interrater and test-retest reliabilities were reported in the original studies. Two broad diagnostic subgroups were examined: psychotic disorder and mood disorder. In classifying participants into diagnostic subgroups, we excluded 23 participants with other disorders.

Substance Abuse

During the study period, case managers rated the Alcohol Use Scale (AUS) and Drug Use Scale (DUS) every 6 months (in Chicago Study, every 3 mo) based on the worst period of substance use in that period, using multiple sources of information (eg, participant self-report, collateral reports, behavioral observations, urinalysis results). The AUS and DUS are rated on a 5-point scale ranging from 1 (“no use”) to 5 (“severe dependence”) and have high reliability and validity when used by trained clinicians.⁴⁰ A score of 3 (“abuse”) or higher on either the AUS or DUS was used as cutoff for presence of substance abuse.

Symptoms

The NH and DC studies measured symptoms using the expanded Brief Psychiatric Rating Scale (BPRS),⁴¹ a 24-item, semistructured interview that rates psychopathological symptoms present during the previous 2 weeks. Each item is rated on a 7-point scale ranging from 1 (“not present”) to 7 (“extremely severe”). The Hartford and Chicago studies measured symptoms over the past 2 weeks using the Positive and Negative Syndrome Scale (PANSS),⁴² a 30-item scale using similar procedures. Good-to-excellent interrater reliability was obtained in the original studies for both scales.

Because the original 18-item BPRS⁴³ is embedded in the PANSS, the current study used scores on the 18 common items to measure symptom severity across RCTs. A factor analysis of the BPRS⁴⁴ was used to define 3 symptom clusters: thought disturbance, anergia, and disorganization. Scores of items on each factor were averaged to derive subscale scores. We also examined the BPRS depression item. The sum of all item scores was used for the overall symptomatology.

For overall symptomatology, a median split was used as cutoff for “high” and “low” subgroups. For the symptom subscales, a score of 4 (“moderate”) or higher on at least one subscale item was used as cutoff for clinically significant symptomatology.⁴⁵ A score of 4 or higher was used as the cutoff for clinical significance on the depression item.

Hospitalization

Two subgroups defined based on hospitalization during the year prior to study admission.

Outcome Measures.

Competitive employment was defined as employment in integrated work settings in the competitive job market at prevailing wages with supervision provided by personnel employed by the business. In the Chicago Study, individual placements, defined as agency-contracted community jobs, were included as competitive employment.³¹ Three competitive employment outcome measures were examined: job acquisition at any time during the 18-month follow-up, total weeks worked in any competitive job during follow-up, and job tenure, defined as total weeks worked on the longest held competitive job during follow-up.

Statistical Analyses

We used Hedges's g as the effect size statistic for total weeks worked and job tenure.⁴⁶ For job acquisition, a dichotomous outcome variable, we first computed the odds ratio effect size, transformed into the natural log, and then rescaled to make the statistic directly comparable to a standardized mean difference effect size.⁴⁷ Effect sizes were calculated for each program site to help minimize confounding variables affecting employment outcome, such as regional economic conditions and fidelity of IPS implementation.⁴⁷ Site-level effect sizes were then used to calculate the overall mean effect size.

In the Hartford Study, a preliminary set of analyses comparing the 2 control conditions demonstrated a lack of statistically differences on the 3 outcome measures. Effect sizes for these comparisons were small: 0.24 for job acquisition, 0.16 for total weeks worked, and 0.17 for job tenure. Consequently, we combined data from the 2 control conditions to create a single control condition.

Homogeneity of each effect size distribution was examined based on a random effects model to assess the moderation effect of program site. This homogeneity test compares the observed variability in effect sizes with an estimate variance that would be expected from subject-level sampling alone.⁴⁶ We obtained a significant Q statistic for 4 (10%) of the subgroups on job acquisition, 11 (28%) for weeks worked, and 11 (28%) for job tenure.

High-resolution forest plots (available from the firstauthor) were evaluated for all meta-analyses. Visual inspection of the distributions of effect sizes suggested that effect sizes for one study site were outliers. The effect sizes for this site were generally smaller and in some cases in the opposite direction than the remaining sites. Reanalyzing the data with this site removed eliminated all statistically significant Q statistics and otherwise yielded a similar pattern of results. In this article, we have opted to report the findings for the full sample.

Meta-analyses were performed using Comprehensive Meta-analysis Version 2.0.⁴⁸ We used descriptive labels for characterizing effect sizes as follows: “small” for values less than 0.33, “medium” for values between 0.33 and 0.55, and “large” for values greater than 0.55.⁴⁹

Results

Sample characteristics comparing IPS and controls in the combined sample are shown in table 2. As expected, the 2 samples were well matched. The only exception was the finding of significantly more severe symptoms for controls, which, however, reflected a small effect size (d = 0.20), not considered clinically significant. The Hartford Study purposively sampled for Latinos and randomly assigned to the 3 groups (IPS and 2 control groups), thereby accounting for the higher proportion of Latinos among controls in the combined sample.

Table 2.

Summary Statistics Comparing IPS Sample (N = 307) to Control Sample (N = 374)

Variable	Overall	IPS	Control	Statistic	P
Work history
Paid work in the community during past 5 y (wk)	Mean (SD)	Mean (SD)	Mean (SD)
	52.43 (69.68)	52.71 (66.98)	52.20 (71.91)	t679 = 0.10	.924
Demographic characteristics
Age	39.53 (9.02)	39.29 (8.84)	39.73 (9.18)	t679 = −0.64	.523
	n (%)	n (%)	n (%)
Gender				χ2 (1, N = 681) = 0.04	.841
Male	372 (54.6)	169 (55.0)	203 (54.3)
Female	309 (45.4)	138 (45.0)	171 (45.7)
Race/ethnicity				χ2 (3, N = 681) = 9.80	.020
Caucasian	265 (38.9)	130 (42.3)	135 (36.1)
African American	311 (45.7)	144 (46.9)	167 (44.7)
Latino	85 (12.5)	27 (8.8)	58 (15.5)
Other	20 (2.9)	6 (2.0)	14 (3.7)
Education level				χ2 (2, N = 681) = 5.47	.065
<High school	229 (33.6)	90 (29.3)	139 (37.2)
=High school	212 (31.1)	97 (31.6)	115 (30.7)
>High school	240 (35.2)	120 (39.1)	120 (32.1)
Marital status				χ2 (2, N = 681) = 0.59	.746
Never married	457 (67.1)	207 (67.4)	250 (66.8)
Married/living together	43 (6.3)	17 (5.5)	26 (7.0)
Divorced/separated/widowed	181 (26.6)	83 (27.0)	98 (26.2)
Disability benefits				χ2 (3, N = 677) = 2.82	.420
SSI only	279 (41.2)	116 (38.0)	163 (43.8)
SSDI only	174 (25.7)	84 (27.5)	90 (24.2)
SSI and SSDI	93 (13.7)	46 (15.1)	47 (12.6)
Neither	131 (19.4)	59 (19.2)	72 (19.4)
Homelessness during past year				χ2 (1, N = 470) = 1.89	.170
Yes	103 (21.9)	46 (19.3)	57 (24.6)
No	367 (78.1)	192 (80.7)	175 (75.4)
Clinical characteristics
Primary diagnosis				χ2 (2, N = 681) = 0.48	.786
Psychotic disorder	429 (63.0)	192 (62.5)	237 (63.4)
Mood disorder	229 (33.6)	106 (34.5)	123 (32.9)
Other	23 (3.4)	9 (2.9)	14 (3.7)
Substance abuse				χ2 (1, N = 651) = 0.08	.772
Yes	106 (16.3)	47 (15.8)	59 (16.7)
No	545 (83.7)	250 (84.2)	295 (83.3)
Symptoms	Mean (SD)	Mean (SD)	Mean (SD)
BPRS thought disturbance	1.93 (0.96)	1.86 (0.95)	1.99 (0.97)	t679 = −1.82	.069
BPRS anergia	1.73 (0.79)	1.68 (0.78)	1.78 (0.80)	t679 = −1.68	.094
BPRS disorganization	1.61 (0.62)	1.56 (0.58)	1.64 (0.65)	t679 = −1.59	.111
BPRS depression	2.45 (1.63)	2.29 (1.58)	2.58 (1.65)	t679 = −2.30	.022
BPRS total	33.21 (9.06)	32.21 (8.51)	34.03 (9.42)	t679 = −2.62	.009
Days hospitalized during past year	19.09 (43.04)	19.32 (43.68)	18.90 (42.57)	t677 = 0.13	.899

Note: IPS, Individual Placement and Support; SSI, supplemental security income; SSDI, Social Security Disability Insurance; BPRS, Brief Psychiatric Rating Scale.

We first examined differences between IPS and controls in the merged dataset at 18-month follow-up, as shown in table 3. The findings strongly favored IPS, with large effect sizes for the 3 employment outcomes: 0.96 for job acquisition, 0.79 for total weeks worked, and 0.74 for job tenure.

Table 3.

Comparisons of Competitive Employment Outcomes at 18 months

	IPS (N = 307)	Control (N = 374)	Statistic	P
Job acquisition	216 (70.4%)	91 (24.3%)	χ2 (1, N = 681) = 144.27	<.001
Total weeks worked	20.53 (24.56)	5.24 (13.94)	t679 = 9.70	<.001
Job tenure (wk)	17.43 (18.53)	4.58 (12.65)	t679 = 9.13	<.001

Note: IPS, Individual Placement and Support.

As shown in tables 4 and 5, the findings were consistent across all consumer characteristics in several aspects. First, in every instance, the effect size was positive. In other words, in every demographic and clinical subgroup examined, IPS participants had better competitive employment outcomes than controls. Second, the large majority of the 117 individual statistical tests were significant at P < .001; all but 8 of these comparisons are significant at P < .01. Third, the effect sizes were nearly all moderate to large in size. Using an effect size of 0.70 as a criterion, of the 39 effect sizes calculated for each outcome measure, 38 (97%) for job acquisition, 28 (72%) for weeks worked, and 24 (62%) for job tenure were large. The ranges of effect size values were as follows: 0.67–1.42 for job acquisition, 0.50–1.06 for weeks worked, and 0.47–1.09 for job tenure. Fourth, the side-by-side comparisons between subgroups within a client characteristic revealed few instances in which one subgroup benefited far more from IPS than the other. For example, those with no prior work history benefited more from IPS for job acquisition while benefiting less for total weeks worked and job tenure. A few exceptions did emerge. For example, participants with more than a high school education benefited less than those with less education, and those who were divorced or separated fared relatively poorly.

Table 4.

Mean Effect Sizes Within Work History and Demographic Subgroups

Client Subgroup	Job Acquisition	P	Total Weeks Worked	P	Job Tenure	P	Mean of 3 Outcomes
Work history
Paid work in 5 y
Yes	1.06	.000	0.83	.000	0.78	.000	0.89
No	1.25	.000	0.63	.000	0.60	.000	0.83
Weeks on paid work in 5 y
<Median (<17.3 wk)	1.03	.000	0.86	.000	0.80	.000	0.90
>Median (>17.3 wk)	1.22	.000	0.66	.000	0.65	.000	0.84
Demographic characteristics
Age
<45 y (N = 495)	1.07	.000	0.74	.000	0.70	.000	0.84
45 y or older (N = 186)	1.18	.000	0.75	.000	0.70	.000	0.88
Gender
Male	1.16	.000	0.84	.000	0.84	.000	0.95
Female	0.97	.000	0.68	.000	0.69	.000	0.78
Race/ethnicity
Caucasian	1.04	.000	0.72	.000	0.66	.000	0.81
African American	1.21	.000	0.82	.000	0.80	.000	0.94
Latino	0.84	.017	0.83	.000	0.84	.000	0.84
Education level
<High school	1.42	.000	1.06	.000	0.93	.000	1.14
=High school	1.26	.000	0.83	.000	0.78	.000	0.96
>High school	0.67	.000	0.51	.000	0.47	.000	0.55
Marital status
Never married	1.10	.000	0.75	.000	0.72	.003	0.86
Married/living together	1.13	.023	0.94	.053	1.09	.029	1.05
Divorced/separated/widowed	0.98	.000	0.50	.055	0.49	.061	0.66
Disability benefits
SSI only	1.24	.000	0.70	.000	0.68	.000	0.87
SSDI only	1.01	.000	0.91	.000	0.83	.000	0.92
SSI and SSDI	1.24	.000	0.89	.001	0.82	.003	0.98
Neither	0.84	.000	0.89	.000	0.82	.000	0.85
Homelessness during past year
Yes	1.13	.000	0.95	.000	0.89	.000	0.99
No	1.11	.000	0.60	.000	0.58	.000	0.76

Note: SSI, supplemental security income; SSDI, Social Security Disability Insurance.

Table 5.

Mean Effect Sizes Within Clinical Subgroups

Client Subgroup	Job Acquisition	P	Total Weeks Worked	P	Job Tenure	P	Mean of 3 Outcomes
Primary diagnosis
Psychotic disorder	1.10	.000	0.79	.000	0.74	.000	0.88
Mood disorder	0.99	.000	0.68	.000	0.66	.000	0.78
Symptoms (BPRS)
<4 on all thought disturbance items (n = 417)	1.01	.000	0.81	.000	0.76	.000	0.86
4 or more on at least one thought disturbance item (n = 264)	1.13	.000	0.68	.000	0.65	.000	0.82
<4 on all anergia items (n = 500)	1.12	.000	0.82	.000	0.78	.000	0.91
4 or more on at least one anergia item (n = 181)	0.99	.000	0.51	.018	0.46	.032	0.65
<4 on all disorganization items (n = 561)	1.05	.000	0.75	.000	0.69	.000	0.83
4 or more on at least one disorganization item (n = 120)	1.20	.000	0.73	.002	0.75	.001	0.89
<4 on depression (n = 481)	0.98	.000	0.72	.000	0.66	.000	0.79
4 or more on depression (n = 200)	1.24	.000	0.78	.000	0.76	.000	0.93
BPRS total < median (32) (n = 358)	1.01	.000	0.82	.000	0.79	.000	0.87
BPRS total > median (32) (n = 323)	1.08	.000	0.63	.000	0.57	.000	0.76
Substance abuse
Yes	0.90	.000	0.62	.000	0.65	.000	0.72
No	1.10	.000	0.81	.006	0.78	.004	0.90
Hospitalization during past year
Yes	1.13	.000	0.75	.000	0.71	.000	0.86
No	1.00	.000	0.77	.000	0.73	.000	0.83

Note: BPRS, Brief Psychiatric Rating Scale.

Discussion

Our meta-analysis indicates that IPS produces better competitive employment outcomes for persons with SMI than alternative vocational programs regardless of a range of background demographic, clinical, and employment characteristics. These findings may come as a surprise to program providers and investigators who assume that supported employment is suitable for only a segment of the target population.^18,50 Instead, the data indicate that IPS outcomes generalize broadly to people with SMI, with no clearly contraindicated subgroups. These findings are consonant with the zero exclusion principle. While we may yet identify subgroups that benefit from sheltered workshops, skills training prior to employment, or some other stepwise approach, the burden of proof rests with proponents of these views.

Our meta-analysis provides practical assistance to researchers planning future RCTs of IPS for specific target populations. Specifically, this article includes a table of effect sizes critical for conducting power analyses. For example, the meta-analysis provides guidance on designing a study evaluating a program for middle-aged or older adults.

Rather than additional RCTs comparing IPS to other vocational models, future research should emphasize enhancing the IPS approach for clients who do not benefit from a trial of supported employment.¹³ For example, clients who are unable to maintain employment or work as many hours as they would like may benefit from individualized clinical interventions tailored to specific deficits and barriers to employment. To date, promising augmentations of IPS have focused on cognitive remediation,²⁶ but there are a host of other barriers warranting targeted interventions, such as co-occurring physical illnesses, poor symptom control, the deficit syndrome,⁵¹ criminal justice involvement, and family interference.

This meta-analysis has several distinctive strengths. All 4 RCTs ensured high fidelity to the IPS model. All provided active, highly regarded comparison interventions, substantially better than those most clients receive in routine mental health care.⁵² All used a common protocol, a standardized follow-up period, and the identical set of predictor measures, thereby avoiding the challenges of calibrating different scales. All used intent-to-treat protocols, with modest sample losses. Finally, because of the common protocol, this meta-analysis required much less imputation than previous meta-analyses. For these reasons, the overall results shown in table 3 and the associated effect sizes significantly advance the literature, including the Cochrane Review⁵³ and Twamley's⁷ review, which are now badly outdated. Another recent review⁴ encompasses more IPS studies but provides limited effect size information and lacks the standardization included in the current analysis.

One criticism of the IPS model has been that it is limited to individuals who want to work and therefore excludes a sizeable proportion of the target population who could benefit from employment services.⁵⁰ This view reflects a misunderstanding about IPS. We strongly concur with the recommendations of the Schizophrenia Patient Outcomes Research Team that vocational goals should be part of every client's treatment plan.¹ The IPS model strongly advocates (and employs) assertive outreach, motivational interview techniques, and active encouragement of employment through a variety of venues.⁵⁴ Ultimately, however, the decision to enter an employment program should be the client's; the purpose of the informational group is to enable clients to make informed choices. Supported employment studies that have enrolled clients without regard to the client's interest in working have predictably obtained substantially lower employment rates.⁵⁵ The proportion of clients enrolled in mental health services who have vocational goals has been estimated in numerous surveys to be in the 50%–60% range.⁵⁶ Clients not expressing vocational aspirations often seek other meaningful roles, such as student, homemaker, retiree with hobbies and recreational activities, self-help group member, and/or volunteer worker, among many other possibilities. Among SSDI beneficiaries with psychiatric disabilities not receiving mental health services, the percentage of those who may agree to participate in IPS is probably closer to 20%–30%.¹⁴ Reaching out to those who are currently unengaged in the world of work is crucially important, but currently the field lacks both adequate information about specific predictors of outcome for this subgroup and evidence-based interventions for engaging those who might benefit. Clearly, this is a direction for future research.

A related issue in the current study regards the external validity of the meta-analytic sample to the population of interest. Specifically, are there substantial selection biases that preclude generalizing these results to the broader population of individuals with SMI who could benefit from supported employment? Several points are pertinent. First, these studies were conducted in public mental health settings so that the issue of translating an efficacy trial into an effectiveness trial does not apply here. Second, the exclusion criteria for the current studies are far less stringent than in medication studies or most studies of psychosocial interventions. Third, in one study included in this meta-analysis, the study sample was compared on demographic and clinical characteristics with the total population of clients with SMI served in the state's public mental health system.³² In general, the statistical differences were modest. Fourth, the 66% participation rate among those attending information groups in the 4 studies was relatively high, suggesting good penetration within the target population. Finally, in contrast to the current group of studies, other studies have examined predictors of employment in supported employment programs that recruited clients without regard to their expressed interest in employment. In these other studies, the interpretation of the results is confounded by motivational issues. So, eg, Gold et al⁵⁷ obtained paradoxical findings in one such study: Cognitive variables did not predict whether or not clients obtained employment, but among those who obtained work, cognitive variables were a predictor. One interpretation of these results is that the heterogeneity of the study sample, by including participants who had no genuine interest in employment, masked the findings for the target sample for whom employment was a goal. In summary, the decision to include clients without a vocational goal in analyses of predictors of employment outcomes is a methodological one; our view is that partitioning the population and treating these as 2 distinct groups and 2 separate research questions will lead ultimately to more precise results and better guide the design and improvement of evidence-based practices.

In any meta-analysis, the question could be raised about the clinical significance of the findings. That is, effect sizes are a statistical measure, but how do the findings translate into practical outcomes? First, we note that working has social validity; little explanation is required for a layperson to understand the meaning of employment, in contrast to, eg, interpreting the meaning of changes on a symptom measure or a cognitive assessment. Second, several prospective studies have suggested that job tenure is associated with a variety of positive outcomes outside the realm of employment, including higher self-esteem, better control of psychiatric symptoms, and less social disability.^58–60 Third, economic modeling suggests that over the long term, small increases in employment across a large population may lead to greatly reduced societal costs.¹⁴

Our meta-analysis has several limitations. (a) Findings are limited to individuals with SMI who are enrolled in mental health treatment. Many other individuals are not in the service system.⁴¹ (b) The 4 RCTs included in the analyses were all conducted by research teams that included the developers of the IPS model. Moreover, the findings apply only to high-fidelity IPS programs and may not generalize to other supported employment approaches. (c) Our findings are also limited to those who want to work. (d) We examined only 3 employment outcome indicators. Future studies should include a broader range of employment variables, including earnings, time to first job, and rates of full-time employment. (e) The 18-month follow-up period was relatively brief. A longer follow-up period might reveal sharper differences because a number of long-term follow-up studies suggest increased employment over time. (f) The measure of job tenure used in this meta-analysis was censored. That is, some participants working at the 18-month follow-up period continued to work beyond that time period, leading to an underestimate of the mean job tenure for the sample as a whole. The best solution to this problem of underestimation is to conduct studies with longer follow-ups. (g) Not all variables of interest were examined in these analyses (eg, neurocognitive variables and criminal justice involvement were not measured). (h) While the sample size was large enough to examine subgroups on individual predictor measures, we had insufficient power to examine combinations of client characteristics (eg, African American males). (i) Finally, the analyses included a large number of statistical tests, leading to alpha inflation. However, this criticism is only a minor concern because the focus was on effect sizes and not statistical significance.

Conclusions

Evidence-based supported employment for individuals with SMI is a flexible model that benefits a wide range of client subgroups compared with active vocational services using conventional ideas of stepwise preparation and brokered services. The current study is one of the first to examine client subgroups with sufficient statistical power.

Funding

National Institute of Mental Health (NIMH) (MH00839, MH00842, MH56147, and R01MH59987); NIMH and the Substance Abuse and Mental Health Services Administration (SAMHSA) (MH47650 and MH51346); National Institute of Disability and Rehabilitation Research (H133G990155); US Department of Health and Human Services SAMHSA Center for Mental Health Services (UD7SM51818).