Effectiveness of vocational interventions for gaining paid work for people living with mild to moderate mental health conditions: systematic review and meta-analysis

Abstract

Objectives

To evaluate the effectiveness of vocational interventions to help people living with mild to moderate mental health conditions gain paid work.

Methods

Systematic review of international, peer-reviewed literature. Development of the prepublished protocol and search strategy was done in consultation with stakeholder reference groups consisting of people with lived experience of long-term conditions, advocates and clinicians. We searched academic databases MEDLINE, EMBASE, PsychINFO, AMED, CINAHL, Proquest Dissertations and Theses database, and Business Source Complete for controlled trials comparing a specific vocational intervention against a control intervention or usual care, published between 1 January 2004 and 1 August 2019. Two authors independently screened search results, extracted data and appraised studies using the Cochrane risk of bias tool.

Results

Eleven studies met inclusion criteria. Seven studies investigated Individual Placement and Support (IPS) modified for people who were not in intensive mental health treatment services. These studies occurred settings such as community vocational rehabilitation services, a housing programme and community mental health services. The studies provided very low quality evidence that people who receive IPS-style vocational rehabilitation are more likely to gain competitive employment than people who receive usual care (risk ratio 1.70, 95% CI 1.23 to 2.34, seven studies, 1611 participants). The remaining four studies considered cognitive behavioural therapy or specific vocational rehabilitation interventions designed to fit a unique context. There was insufficient evidence from these studies to draw conclusions regarding the effectiveness of non-IPS forms of vocational rehabilitation for people with mild to moderate mental health conditions.

Discussion

The meta-analysis showed a clear intervention effect but low precision, and more high-quality studies are needed in this field. There is currently very low quality evidence that IPS-style intervention results in more participants in competitive employment compared with ‘usual care’ control groups in populations with mild to moderate mental health conditions.

Strengths and limitations of this study.

• This is the first systematic review and meta-analysis looking at interventions to support people living with mild to moderate mental health conditions to gain paid work.

• A rigorous, prepublished systematic review protocol was used, informed by stakeholder consultation.

• Conclusions are limited by the number of studies and the methodological quality of those studies.

Introduction

People who experience chronic health conditions and/or disability often have difficulty accessing paid work due to a number of complex factors. These factors include stigma, poorly informed judgements about what makes somebody ‘work-able’, fear of negatively affecting health and safety in the workplace or making the workplace vulnerable to higher costs, and challenges in negotiating work arrangements that are outside the norm when needed.^1–5 Case management, vocational rehabilitation and employment support can all help to address these challenges to improve work outcomes for people experiencing health conditions and/or disability, and various approaches exist within these broad categories that have been tested in research.

Supported employment is a type of vocational rehabilitation that focuses on securing paid work for people then providing them support to become competent in that role while working.⁶ This is in contrast to an approach that might instead delay entry into the workplace until people are deemed to be fully trained and independent workers. In mental health, a particular approach to supported employment has been developed for people diagnosed with severe mental health conditions—Individual Placement and Support (IPS).⁷ IPS is characterised by several standardised principles, which can be measured using a fidelity scale.^8–10 A summary of the main principles of IPS are as follows:

• IPS is provided by employment specialists with low caseloads (20 or fewer).

• No exclusion criteria—IPS is provided to any people who wants a job.

• Integrated with mental health treatment—employment specialists are treated as a discipline within a mental health multidisciplinary team.

• Employment specialists work with clients across all phases of their vocational journey.

• Integrated with financial advice and work incentives planning.

• Individualisation of job searches to each person’s interests and skills.

• Rapid job search for competitive employment.

• Employment specialists are actively engaged in job development on behalf of their clients (relationships with employers, actively seeking to create individualised opportunities).

• Ongoing support once working, as appropriate to each situation.

While IPS has been designed for people in intensive treatment services for mental health and addiction, positive outcomes from this approach have prompted adaptation into other populations.¹¹ The adaptations have varied according to context, but include locating the employment specialist within a team that is focused on interventions other than mental health treatment,^{12 13} and/or the addition of more explicit empowerment components.^{12 14}

Several systematic reviews have been published on the effectiveness of vocational interventions for people with severe mental health conditions such as severe depression, bipolar disorder and psychotic conditions.^15–24 The most recent of these reviews was a network meta-analysis,²⁴ which met almost all of the criteria of the AMSTAR-2 assessment tool for assessing the quality of systematic review.²⁵ This network meta-analysis concluded that there was low quality evidence that supported employment was effective in aiding people diagnosed with severe mental health conditions to gain competitive employment (relative risk (RR) 2.72, 95% CI 1.55 to 4.76) and moderate quality evidence for augmented supported employment (RR 3.81, 95% CI 1.99 to 7.31) when compared with psychiatric care alone. To the best of our knowledge, however, no systematic reviews have been conducted on vocational interventions for people with mild to moderate mental health conditions.

Mild to moderate mental health conditions are common globally.²⁶ Additionally, some people experience mental health symptoms that are subthreshold for diagnosis. These people are at risk of being missed in service delivery.²⁷ Mental health conditions, particularly anxiety and depression, are a major concern as contributors to work incapacity,²⁸ but people with mild to moderate depression and anxiety are not typically included in clinical trials that test vocational rehabilitation or employment support interventions designed specifically for mental health populations. Arguably, the most effective way of preventing work disability for people experiencing mild to moderate mental health conditions is to provide support to maintain existing employment, as psychosocial issues within and outside of the workplace have been shown to be significant predictors of failure to return.²⁹ Furthermore, long periods away from work lead to less likelihood of return.²⁹ However, it is common for people to be out of work before they seek support, and often the support provided is done through social insurance agencies that also manage unemployment benefits. Because of this, effective interventions that are designed to support people to gain and maintain new employment are vital. The aim of this systematic review therefore was to evaluate the effectiveness of interventions for providing support for gaining employment to unemployed people with mild to moderate mental health conditions.

Consultation with stakeholder reference groups

Although systematic reviews are often seen to be ‘objective’, they are not impartial, as the design of the search strategy and assessment of relevance affects what literature is accessed and included.³⁰ Further to this, the way in which findings are reported and made relevant to the local context is crucial in how the information is used in practice. Explicit stakeholder involvement in systematic review processes to address external validity and transferability of findings is a recent development in systematic review design.³¹ Our methods included explicit involvement of stakeholders in informing the design, contextualisation and reporting of the review.

Review questions

1. What is the effectiveness of vocational interventions to help adults with mild to moderate mental health condition(s) gain and maintain paid work in comparison to no treatment, an attention control or usual vocational services?

2. What is the quality of evidence on vocational interventions to help adults with mental health condition(s) gain and maintain paid work?

3. What evidence gaps exist in relation to vocational interventions to help people with mild–moderate mental health condition(s) gain and maintain paid work?

Methods

For this review, we followed the methods recommended by the Cochrane Handbook for Systematic Reviews of Interventions.³²

Patient and public involvement

Stakeholder reference group consultation

At three key stages in the process—protocol development, considering transferability and external validity of the evidence profile and dissemination of findings—we consulted with two stakeholder reference groups. The condition-specific stakeholder reference group was made up of people with lived experience of mild to moderate mental health conditions and people involved in services and advocacy in this field. We also consulted a specific Māori stakeholder reference group (indigenous culture of Aotearoa New Zealand in which the review was undertaken) that included people with professional and personal experience across a range of long-term conditions. The reason for having a separate Māori group is that Indigenous perspectives are often overlooked and/or under-represented in mainstream research. The protocol and search strategy for this study was refined with the stakeholder groups and prepublished before we searched the literature.³³ The stakeholder reference groups were involved again later to provide input into contextualising findings for the local context and peoples, and assisted with various methods of dissemination in the community including meetings, discussions, webinars and user-friendly summaries.

Criteria for considering studies for this review

Type of studies

We included randomised controlled trials (RCTs), cluster RCTs, pseudo-RCTs and non-RCTs that assess the effects of vocational interventions for unemployed people with mild to moderate mental health conditions. We included published, peer-reviewed literature and theses and dissertations. We excluded studies that were not published in English. We included the various types of controlled trials so as to be inclusive as possible, anticipating a low number of studies due to this being a relatively new area.

Type of participants

We included studies where participants were 16 years of age or older, who had a mild to moderate mental health condition and were unemployed at study outset. We defined a mild to moderate mental health condition as living with symptoms that indicate a mild to moderate mental health condition (eg, anxiety, depression) regardless of whether or not the person had been given a formal diagnosis. To operationalise this definition, we excluded diagnoses that frequently require specialist intervention (eg, inpatient, community mental health team) such as bipolar, schizophrenia and psychotic conditions. More commonly, mild to moderate mental health conditions such as mild to moderate anxiety and depression are either managed in general practice or go undiagnosed.

Because of the difficulty in capturing mild–moderate severity using keywords, we did not limit the search by severity, but excluded studies that focused on ‘severe’ diagnoses at the assessment of relevance stage. Where studies included a mix of participants who met our definition of mild–moderate severity and also participants with greater severity, we included the study if the proportion of participants meeting our inclusion criteria was 60% or greater. We based this decision on an assumption that where the intervention is appropriate for a range of severities, participants with more severe diagnoses are more likely to affect outcomes negatively rather than positively.

For the purpose of this review, participants were considered unemployed if they were not in paid work. We defined paid work according to the Resolution concerning statistics of the economically active population, employment, unemployment and underemployment, adapted by the Thirteenth International Conference of Labour Statisticians (October, 1982)³⁴ or as a legal occupation that generates a livelihood (eg, Indigenous practices that generate resources to live on but are not paid employment).

Type of interventions

Employment support services considered in scope included the following:

• Training or education to prepare people to apply for employment (eg, preparation of resumes; development of work skills; job-specific skills training and licencing (eg, drivers licence), training in employment interview techniques; motivational and behavioural interventions to help people increase their self-efficacy for employment and engage in work).

• Vocational counselling to help people identify their work skills and aptitude, seek work, or find and apply for jobs.

• Provision of support for people to apply for jobs or actively seek employment in other ways (including peer support programmes).

• Identification of environmental barriers to employment (eg, workplace, transport or attitudinal barriers) and implementation of supports and/or strategies to reduce or remove those barriers (eg, job accommodation, workplace modification—including access, provision of equipment, employer education, supporting workplace relationships and other psychosocial interventions, negotiating health and safety requirements, providing adapted transport).

• Job development (eg, working with employers to create non-standard or modified positions).

• Job placement schemes.

• Provision of support for people newly in employment.

• Provision of financial and related supports to move into employment (eg, appropriate clothing, equipment, transportation, transitional loans).

• Job coaching (short or long-term) to develop skills and strategies to manage a job.

• Community development specifically focused on creating employment opportunities for people living with long-term conditions and/or disabilities (eg, development of business initiatives that need employees with lived experience of disability, employer awareness, incentive programmes, social enterprises focused on work for this population).

• Community and family support initiatives focused on enabling participation in paid work.

• Work-readiness training or transitional employment where the focus was on transitioning into paid employment or providing skills for people to immediately enter paid employment at the completion of the training.

We excluded studies that focused on stay-at-work interventions for people who are currently in paid work and needing to maintain that work in the context of mild to moderate mental health conditions. We excluded sheltered workshops where the expectation is that participants will earn less than minimum wage and retain social welfare benefits for the foreseeable future. We also excluded interventions that were focused on helping people to gain or maintain occupational roles and activities that are unpaid and also unlikely to provide a livelihood.

Type of comparisons

We included studies with a no intervention control, an attention control (ie, a Hawthorne control) or a comparison to ‘usual care’ as defined by the study authors. We also included studies that compared two different approaches to vocational rehabilitation.

Types of outcomes

Primary outcomes

The main outcomes for the review were (1) commencement of either full-time or part-time paid work; (2) successful maintenance of new paid work for 6 months or longer.

We extracted data on rates of work placement in any type of paid work, and whether these placements were full-time or part-time; casual, fixed term or permanent (where reported). We did not consider gaining an employment contract alone to constitute commencing work if no paid work was carried out. We extracted data on maintenance of new work using count data (eg, number of people maintaining employment at 6 months) and continuous data (eg, duration of time employment has been maintained by the end of a study) as reported by the study authors. We considered outcomes at three time points: the end of intervention (as reported by study authors); up to 1 year after the end of an intervention; more than 1 year after end of an intervention.

Secondary outcomes

Where reported, we also extracted data on the following outcomes: proportion of full time work, satisfaction of employers with employment outcomes or work performance, satisfaction of participants with work outcomes, pay rates, cost of vocational interventions, cost of ongoing work support, quality of life, cost–benefit analyses, community participation, financial autonomy, self-esteem, self-efficacy, effect on health status, tensions between work expectations and requirements for rehabilitation, treatment or healthcare needs (eg, time to go to specialist appointments). We have not reported on these in this paper as there was insufficient data to draw conclusions.

Study identification and selection

We searched published literature and theses and dissertations located in seven databases: MEDLINE (OvidSP), EMBASE (OvidSP), PsychINFO (OvidSP), AMED (OvidSP), CINAHL (EBSCOhost), Proquest Dissertations and Theses database and Business Source Complete (EBSCO). Search strategies were designed and tailored to each database using key words and subject heading terms. The detailed search strategy for MEDLINE was included as an appendix to the prepublished protocol.³³ Database searches covered 1 January 2004 to 1 August 2019. This was that timeframe that authors considered to be appropriate for ensuring relevancy of policies and systems relating to vocational rehabilitation.

Covidence software was used to manage the selection process (Veritas Health Innovation, Melbourne, Australia, www.covidence.org). Two review authors per study (from JF, DA and KR) independently considered each study against prespecified inclusion criteria. Initially, titles and abstracts from the identified studies were screened and studies that may meet inclusion criteria were included for full-text review. Disagreement or uncertainty about relevance was resolved through consideration and discussion of full study reports, involving an additional reviewer where necessary.

Data extraction

Two authors extracted the data from all included studies into a prespecified form (DA and MK) which was then checked by a third author, who prepared the summary tables (JF). We extracted data on study population and context, methods, interventions and their implementation and primary and secondary outcomes for our review.

Risk of bias assessment

We used the Cochrane risk of bias tool³⁵ to assess the quality of each included study. This is a seven-item tool for assessing the risk that the results of in a controlled trial are subject to bias from aspects of the study design or execution. We scored ‘yes’, ‘unclear’ or ‘no’ for each item and the risk of bias as ‘low’, ‘unclear’ or ‘high’ for each study overall. Study authors were contacted for clarification, and responses included in assessment if we received a response.

Data synthesis

In our prepublished protocol, we only described methods for reporting on study findings descriptively because we had not anticipated finding sufficient studies of a similar nature to conduct a meta-analysis. However, as we found a number of similar studies that tested the effectiveness of interventions based on an IPS approach, we change our methods to include a meta-analysis of this evidence.

We conducted the meta-analysis according to methods in the Cochrane Handbook for Systematic Review of Interventions.³⁵ We analysed dichotomous data related to successful employment outcomes using RRs. We conducted our analysis on the basis of per-person randomisation. All included studies were parallel RCT designs. We did not include data from cluster or cross-over RCTs. Where studies had more than two participant groups, or analysed subgroups separately, we only included data from the groups or subgroups relevant to our review question. For studies where data were collected at more than one time point, we only included data from the furthest time point from the studies’ enrolment. We used the I² statistic to measure heterogeneity among the trials in each analysis. We did not have enough studies to explore causes of heterogeneity by any subgroup analysis. We also did not have enough studies to use funnel plots to explore small-study and publication biases. We undertook sensitivity analyses to examine the influence of risk of bias associated with including the studies in each meta‐analysis. We removed studies at the greatest risk of bias (ie, those that failed to randomise adequately or failed to conceal random allocation) from the analysis in order to test the strength of evidence for the various effect estimates.

We used GRADEPro to evaluate the quality of evidence arising from this review.³⁶ Following the GRADE approach, we upgraded or downgraded the quality of evidence based on assessment of risk of bias, inconsistency, indirectness, imprecision and other considerations such as publication bias, size of effect, evidence of any dose response gradients and plausible confounding variables.

Results

Results of the search

As illustrated in figure 1, we identified 4828 citations from the database searches in total. Of these, 3255 remained after removal of duplicates (n=1573). After we had screened all titles and abstracts, we identified 33 potentially relevant studies in full text. Of these 33 titles, we included 11 studies and excluded 22. The reason for these exclusions are listed in online supplemental studies excluded at full text review.

Figure 1.

Preferred Reporting Items for Systematic reviews and Meta-Analyses flowchart.

Characteristics of included studies

The 11 included studies included a combined total of 2416 participants in the groups relevant to our review. These studies were conducted in the USA (n=4) Sweden (n=2), Norway (n=2), Netherlands (n=1), Denmark (n=1) and Canada (n=1). Three of the studies from the USA were focused specifically on an armed forces veteran population. The risk of bias profile across the included studies is shown in figure 2. Three studies^{12 27 37} were evaluated as being of low risk of bias, four studies^{14 38–40} of unclear risk of bias and four studies^{13 41–43} of high risk of bias.

Figure 2.

Risk of bias assessment by study

The included studies covered a range of interventions that we grouped into three categories:

• IPS approaches modified for people who are not in intensive mental health treatment.

• Cognitive behavioural therapy with work-focused components.

• Other interventions.

Description and synthesis of study findings

IPS approaches

Seven studies investigated the effectiveness of IPS interventions including two studies of low risk of bias,^{12 27} two of unclear risk of bias^{14 39} and three of high risk of bias.^{13 42 43}

The particular context varied for each study, as did the adaptations made for the different population. All studies tested IPS in a mild–moderate mental health condition population, which differed from the population IPS was originally designed for. In most studies, the setting was also different—that is, it was not delivered in a specialist mental health treatment setting. In some studies, the study team also slightly adapted the IPS intervention principles. However, in each case, the core principles of IPS were adhered to, and existing IPS fidelity scales were used. Most of the reported studies used an IPS fidelity scale to assess both the intervention and control condition, demonstrating that the control condition achieved a score that indicated ‘not-IPS’. The longest reported follow-up point ranged from 8 months to 24 months from beginning the intervention, with the median being 18 months.

See table 1 for details on the characteristics of the seven studies testing IPS interventions, and table 2 for further information on the implementation of these IPS interventions in each study context.

Table 1.

Study characteristics for IPS interventions in mild–moderate mental health populations

Reference	Population	Intervention	Comparison	Fidelity information
Bejerholm et al, 201714	Depression or bipolar (majority this sample depression)	Individual Enabling Support (IES) (principles 3–10 as for IPS) (n=33)	Traditional vocational rehabilitation—‘train then place’ (n=28)	Good fidelity as measured at 6 and 12 months using Supported Employment Fidelity Scale (SEFS) 20088 (delivery score 106–109)
Davis et al, 201243	Veterans with Post Traumatic Stress Disorder	IPS (n=42)	Usual care in Veterans Health Administration Vocational Rehabilitation Programme (n=43)	Fidelity on SEFS 19979 fair (55–65) throughout study
Davis et al, 201842	Veterans with Post Traumatic Stress Disorder	IPS (n=271)	Transitional work (n=270)	Fidelity on SEFS 19979 poor (55) in first 3 months, but good fidelity (63–69) maintained after that
Hellström et al, 201712	Affective disorder or anxiety, no mental health service last 3 years	IPS modified for mood and anxiety disorders (IPS–MA) (n=162)	Usual services offered by job centres in Denmark (n=164)	High fidelity throughout study on IPS-259 (100–103)
Poremski et al, 201713	Recently homeless, with mental illness	IPS (n=44)	Community-based services including some case manager support (n=41)	Fidelity on IPS-2510 initially fair (73) and gradually increased. Good (100) by final 8 months of intervention
Reme et al, 201527	Mainly anxiety and/or depression, including subthreshold for diagnosis	At work and coping: work-focused cognitive behavioural therapy alongside IPS adapted for a population with mild–moderate mental health conditions (n=177)	Standard services from health professionals and national insurance office (n=177)	Fidelity tested postrecruitment using IPS-25.9 Reported as achieving ‘adherence to IPS’, no individual scores given
Reme et al, 201939	Diagnosed psychiatric disorder (moderate severity)	IPS (n=227)	High-quality usual care. Non-IPS interventions included work with assistance or a ‘traineeship’ in a business (n=181)	All centres scored fair, good or exemplary (74–125) on IPS-2510 by second (of three) assessment points and increased scores during the trial. Exact scores not reported

*Fidelity indicates the level of coherence of the intervention as implemented and practiced with the principles of IPS. Scores range from ‘not-IPS’ upward. Various fidelity scales were used. We have reported the scale, the reported interpretation and the scores where available. Possible high scores are 125 for SEFS 2008 and IPS-25; 77 for SEFS 1997.

Table 2.

Implementation context for IPS intervention studies

Reference	Long-term condition	Country	Community and economic context	Intervention team setting	Key implementation information reported
Bejerholm et al, 201714	Affective disorder	Sweden	Swedish context (little information regarding community and economic context). Middle-sized cities, geographically diverse.	Employment specialist (with additional specialist training in cognitive behavioural therapy, motivational interviewing and time-for-work strategies) and participant working closely with outpatient (mental health?) team, family, social insurance agency and public employment service.	This service was delivered in the context of mental health services. The theory appears to be that this population can benefit from leading the process more through motivational and empowerment strategies. Two additional principles specific to the population focus on enabling motivation and creating cognitive strategies characterise the Individual Enabling Support (IES). These are in addition to the usual IPS principles: ‘(1) handling change and developing motivational and cognitive strategies, (2) having a time-use pattern that supports work-life balance.’ (p213)
					‘The extent of the counselling is individualized to fit the intervention need of the participant. IES involves phases of (1) enabling mobilization of motivational, cognitive, and lifestyle strategies, (2) completion of a career profile and plan, (3) job-seeking, and (4) supported employment phase during which mobilized strategies are intertwined. Phases 1 and 2 last for 1–2 months. Phase 3 lasts until employment is reached. Phase 4 is the remaining time. The duration and intensity of phases 1, 2, and 3 are approximately one hour per week, while phase 4 requires 20 min per week.’ (p214)
Davis et al, 201243	Veterans with Post Traumatic Stress Disorder	USA	Armed forces veterans are able to receive compensation from the veteran’s health administration. Medical centres specifically for veterans. Programme delivered through a medical centre in Tucaloosa.	Employment specialist integrated into the veterans mental health treatment team	IPS service was delivered in line with two IPS manuals, but operated as part of the Veterans Health Administration Compensation Work Therapy programme. 25 client maximum caseload. Table 1 (p466) in the article outlines the difference between the IPS service and the usual care service provided through the same programme.
Davis et al, 201842	Veterans with Post Traumatic Stress Disorder	USA	Armed forces veterans are able to receive compensation from the veteran’s health administration (VA). Medical centres specifically for veterans. Programme delivered through these VA medical centres was a stepped programme that included transitional work.	Employment specialist integrated into the veterans mental health treatment team	25 client maximum caseload. Brief reporting, the intervention format appears similar to the 2012 study.
Hellström et al, 201712	Affective disorder or anxiety	Denmark	Job centres in Denmark deliver a variety of vocational support services for people who need to find work. Benefits can be received for a maximum of 52 weeks. In contrast to the usual job centre services, the intervention was delivered by a private company.	Each individual allocated a mentor (minimum 10 years experience in mental health as nurse, occupational therapist or social worker) and a career counsellor.	Eligible participants had not received any metal health services in the last 3 years.
					IPS–MA is detailed in full in the published protocol.48
					‘Briefly, the intervention consisted of mentor support and career counselling, providing five basic services: individualised mentor support based on psychiatric knowledge; coordination of services provided; career counselling; impartial help to clarify private economy (this included benefits counselling if required); and contact with employers to help participants obtain jobs and keep them. Focus was on competitive employment and support was time unlimited. A plan of action was created based on goals, resources and challenges related to work/education, social relations and leisure activities, and the plan was evaluated regularly. Participants had the same mentor throughout the intervention, and support continued for as long as needed. The number and duration of contacts depended on the individual needs; most met with their mentor once a week for 1–1 1/2 hours. Each mentor had a maximum caseload of 20 participants in order to secure the flexibility of the support.’ (p718)
Poremski et al, 201713	Mental illness	Canada	People with recent homelessness. Delivered in the context of a Housing First programme.	Employment specialist (trained by a senior member from a local IPS service) integrated into the Housing First clinical services team	Only reported that they delivered the IPS approach, delivered by an employment specialist located within the Housing First service.
Reme et al, 201527	Mainly anxiety and/or depression, including subthreshold for diagnosis	Norway	Worker’s compensation programme that provides 100% of former income until return to work or up to 52 weeks, and long-term benefits after that averaging 66% of former income. Employment support is an aspect of the services offered. These services are an adaptation of the Individual Placement and Support model (see implementation info). The adaptations were related to the mild–moderate population but also the service delivery constraints.	‘Mini-team’ of team leader (psychologist), employment specialist knowledgeable in IPS and psychological therapist (psychology or similar degree and cognitive behavioural therapy trained).	Eligibility assessment conducted by clinical psychologist. Based on self-report symptoms.
					Participants needed to express motivation to return to work to participate.
					Up to 15 cognitive behavioural therapy sessions were offered in addition to the IPS principle-based approach
					The hypothesis was that the interaction between these two aspects of services for the particular population that would make the difference. The online protocol provides a full breakdown of the interpretation of the principles of Individual Placement and Support in the context of the At Work And Coping intervention: https://oem.bmj.com/content/oemed/suppl/2015/07/24/oemed-2014-102700.DC1/oemed-2014-102700supp_Protocol.pdf
Reme et al, 201939	Diagnosed psychiatric disorder (moderate severity)	Norway	Small, open economy with low unemployment rate. The authors describe the policy context as ‘high job security and a comprehensive welfare system’. High sickness absence rates compared with other OECD countries.	Employment specialist as part of the mental health treatment team.	IPS was delivered in a special ‘IPS centre’. This was in contrast to the usual vocational services which these authors have described in the previous study as IPS-based. Note that this may influence effect size. The protocol describes the principles of IPS and fidelity information is provided. No further detail on implementation is reported.

IPS, Individual Placement and Support.

For the meta-analysis (see figure 3), we used random effects methods because there was substantial heterogeneity in the data (I²=78%). The meta-analysis showed that an IPS approach vocational intervention resulted in an increased likelihood of unemployed people with mild to moderate mental health conditions gaining employment when compared with a usual care approach (RR 1.70, 95% CI 1.23 to 2.34, seven studies, 1611 participants). Removal of studies at the greatest risk of bias (ie, those that failed to randomise adequately or failed to conceal random allocation)^{18 39 44} shifted the pooled risk ratio more in favour of an IPS style of vocational intervention, with a widening of the 95% CI (RR 2.35, 95% CI 1.18 to 4.65).

Figure 3.

Forest plot Individual Placement and Support for employment, mild to moderate mental health.

Following the GRADE approach, we downgraded the quality of evidence underpinning this finding because of very serious problems with inconsistency (due to large statistical heterogeneity) and serious problem with imprecision (due to the wide 95% CIs). This meant that the overall certainty of evidence supporting this finding was very low. The addition of a single large, well-conducted clinical trial in the future could easily change our estimated effect size for this comparison.

Cognitive behavioural therapy focused interventions

Two studies investigated the effectiveness of cognitive behavioural therapy as supplement to vocational services in comparison to physical exercise or treatment as usual for unemployed people with mild–moderate mental health conditions. This included one study of low risk of bias³⁷ and one of unclear risk of bias.³⁸ The low risk of bias study was a US-based pilot study (n=58) where cognitive behavioural therapy was delivered in the context of vocational services.³⁸ The other study was a large Swedish study where cognitive behavioural therapy was delivered via the internet, and included work-related modules.³⁷ Not all participants in the latter study were relevant to our review question, but this study included a subgroup analysis of participants with mild to moderate mental health conditions who were unemployed at the study outset (n=98). The characteristics of these studies are presented in tables 3 and 4. Neither of the studies reported statistically significant results on employment outcomes. The overall certainty of evidence supporting this finding was very low.

Table 3.

Study characteristics for cognitive behavioural therapy focused interventions in mild–moderate mental health populations

Reference	Population	Intervention	Comparison	Statistical power of sample	Reported paid work outcomes
Kaldo et al, 201737	Depression	Internet-based cognitive behavioural therapy (ICBT) (n=38)	1. Physical exercise—60 min, 3x per week (n=40)	Sample size calculation for main study only (sample size achieved). Unemployed at baseline subgroup analysis small sample (n=118)	Employment status not significantly different between ICBT group and treatment as usual comparison or between ICBT group and physical exercise comparison at 12-month follow-up in the subgroup unemployed at baseline.
			2. Treatment as usual—standard care for depression as determined by general practitioner (n=40)
Himle et al, 201438	Social anxiety disorder	Work-related cognitive-behavioural therapy (n=29)	Usual care—this normally included career assessment, job interview skills training and job placement assistance (n=29)	Small sample (pilot study). Not powered to detect differences.	No significant difference between groups in those who worked for pay in 12 weeks prior to follow-up (44 of the total 58 participants, specific numbers by group not reported) condition-related or the work hours of working participants.

Table 4.

Implementation context for cognitive behavioural therapy focused interventions

Reference	Long-term condition	Country	Community and economic context	Intervention team	Key implementation information reported
Kaldo et al, 201737	Depression	Sweden	Participants were recruited from primary care facilities across six counties. Intervention was delivered on the internet. People on a ‘disability pension’ were excluded. It is unclear what this means in a Swedish context.	Support provided for participants using the intervention by a clinical psychologist and a final-year clinical psychology student under supervision.	‘Before treatment, patients received a short phone call from their therapists, who explained the treatment process and helped with technical issues. They also filled in several online questionnaires regarding depression, worry, panic attacks, social anxiety, stress, insomnia, pain and work-related problems.’ There were ‘three introductory modules addressing problems related to depressive symptoms, such as inactivity and avoidance behaviours, the subsequent modules were tailored to the patient’s specific clinical profile, mainly based on the areas mentioned above. In total, 34 modules were available and 30 of these were used to individually tailor the treatment.’ ‘four of the modules aimed at managing problems related to work and sick leave: (one was focused on) finding a new job, which participants without employment could receive this module about the job seeking process and homework assignments about scheduling job seeking.’ (p55)
Himle et al, 201438	Social anxiety disorder (SAD)	USA	Delivered at a vocational rehabilitation centre in Detroit.	Delivered by vocational services professionals who had been trained in the use of cognitive behavioural therapy by specialists in CBT for anxiety disorders (30–50 hours training each plus weekly supervision).	Eight, 2-hour sessions of work-based cognitive behavioural therapy, held twice weekly over the course of 4 weeks in addition to standard vocational services. Delivered concurrently with the vocational services, but scheduled at a different time of day.
					‘Session 1 of WCBT (work-related cognitive behavioural therapy) involves psychoeducation related to SAD and its effect on employment. Session 2 primarily involves instruction in the identification of automatic thoughts. Session 3 involves further discussion about how SAD relates to the world of work and instructs participants in constructing rational responses to their automatic thoughts. Sessions 4–8 include a psychoeducational topic related to the world of work, in-session exposure as well as cognitive restructuring, and homework exercise planning’. (p172)

Other interventions

The remaining two included studies were focused on two different interventions, were designed for very specific population groups and were assessed as having unclear⁴⁰ or high risk of bias.⁴⁵ These studies investigated an intervention focused on ‘activation’⁴⁵ and an incentive-based contingency management programme.⁴⁰ A summary of these studies is presented in table 5.

Table 5.

Summary of study characteristics for other interventions in mild–moderate mental health populations

Reference	Population	Intervention	Comparison	Statistical power of sample	Reported paid work outcomes
Audhoe 201645	Netherlands People identified to have ‘psychological problems’ who were unemployed when ‘sick listed’ or had temporary contracts that will expire.	‘Brainwork’ intervention (n=164)	Usual care—sickness absence counselling (n=156)	Considered calculation. Calculated sample size achieved.	Duration of sick leave lower (171 days) in the intervention group than in the usual care control group (185 days) but difference not significant. Mean hours in paid employment for those working during the 6-month follow-up period was significantly greater in the control group than the intervention group (443 hours vs 257 hours; p=0.005).
Drebing et al, 200740	US Veterans with dual diagnosis (Mental health diagnoses majority moderate).	Incentive-based Contingency Management added to vocational rehabilitation (n=50)	Vocational rehabilitation only (n=50)	Small sample size. No calculation reported.	Significantly more intervention group participants (50%) competitively employed compared with vocational rehabilitation only participants (28%) at 9-month follow-up; p<0.05. Tenure no significant difference, but this analysis was limited by low number in employment and short follow-up period.

Discussion

This systematic review has found very low quality evidence that the IPS approach, modified for people who are not in intensive mental health treatment services, is effective to assist unemployed people living with mild to moderate mental health conditions to gain competitive, paid work. More high-quality studies are needed and further studies are likely to change the estimate of effect. There was inconclusive evidence in relation to cognitive behavioural therapy and other approaches as interventions to support gaining paid work.

Strengths of the review were that it covered a wide range of databases and the protocol and search strategy were informed by consultation with clinicians, advocates and people with lived experience of long-term conditions. The involvement of stakeholder reference groups added value in a number of ways, which is consistent with the increased attention on this practice in recent methodological discussion.³¹ While the groups did not participate in the execution of the systematic review, they had important input into the protocol (which influenced inclusion criteria) and contextualisation of findings. For example, the Indigenous stakeholder reference group advised the research team to broaden the definition of ‘paid work’ in the protocol to include indigenous practices and other activities that generate livelihood that may not be considered ‘employment’. We found this to be an important critique of the ways in which the outcomes of vocational interventions are usually conceptualised. Furthermore, both groups were able to provide extensive discussion on condition-related language and terminology that the research team were able to test out in scoping searches in order to develop an inclusive search strategy. Stakeholder reference groups were also able to provide valuable input for discussing findings in a way that was understandable with reference to both lived experiences of people living with mental health conditions and also existing service provision and funding discourses.

The review was limited by the exclusion of studies that were not reported in English, reliance on databases that index peer-reviewed journals and thesis repositories, and lack of grey literature beyond theses. As such, the conclusions we were able to draw from this systematic review and meta-analysis do not take into account the results of studies that were published in other languages or in sources not indexed in the databases we utilised. The meta-analysis in relation to the IPS approaches showed a clear intervention effect but had low precision, limited by the relatively small number of studies and participants and by large statistical heterogeneity across the studies. Because of these factors, results should be interpreted cautiously. It is also worth noting that the specific intervention settings varied considerably. The justification for meta-analysis was based on the fact that the IPS principles were followed in each situation. However, even where the same principles are implemented, knowledge of the ways in which this is done in different settings is important for clinical application. Implementation case studies and qualitative work is important for understanding the various ways of operationalising the principles. While most of these studies currently are based on the intensive mental health treatment services setting, they offer critical information on real-world implementation. Two recent examples are a critical analysis of implementation of IPS in rural New Zealand⁴⁶ and a longitudinal analysis of the implementation of IPS in an Italian setting, including participants with ‘moderate’ diagnoses.⁴⁷

As far as we are aware, this is the first systematic review to address employment support interventions for unemployed people living with mild to moderate mental health conditions. One recent review by Bond and colleagues¹¹ looked at studies reporting on IPS in any population other than ‘serious mental illness’, and included a meta-analysis. There was considerable overlap in the studies used for Bond et al’s meta-analysis and ours. However, Bond et al did not include one relevant study on vocational rehabilitation for people with mild to moderate mental health conditions¹³ and included three other studies unrelated to this review: two on people with substance use disorder and one on people with spinal cord injury. The estimate from the Bond and colleagues meta-analysis is also in favour of IPS, but with wider CIs (OR 2.23, 95% CI 1.53 to 3.24).

While many existing vocational interventions are currently provided worldwide, with the intent of supporting people into paid work, very few of these interventions have ever been tested in controlled trials. Most of the studies in this review compared the vocational rehabilitation intervention being tested against ‘usual care’. What constituted ‘usual care’ was not always well described, but generally involved some form of case management and referral to services classified as ‘vocational’ such as skills workshops and resume preparation. The critique offered by the success of the IPS model to date is the focus on good matching, job development and actual job placement that is then supported, rather than job preparation that does not necessarily lead to work. While not everyone will require services of the intensity that IPS was designed for, these studies indicate that the principles themselves could be applied in a wider context. There is still considerable work to do to determine how IPS is best adapted for populations other than for people in specialist mental health and addiction services, and the extent to which IPS fidelity indicators need to be amended for less intensive contexts. Further studies will help address these issues. In addition, further large, well-conducted clinical trials are needed to strengthen the evidence on the effectiveness of vocational interventions for people living with mild to moderate mental health conditions, ideally with longer follow-up periods to track the sustainability of employment outcomes over time.

Conclusion

There is very low quality evidence that IPS helps unemployed people living with mild to moderate mental health conditions gain competitive employment over an 8–24 month period. There is currently insufficient evidence to draw conclusions regarding the effectiveness of other forms of vocational rehabilitation for people living with mild to moderate mental health conditions.