Preventing occupational stress in healthcare workers

Jani H Ruotsalainen; Jos H Verbeek; Albert Mariné; Consol Serra

doi:10.1002/14651858.CD002892.pub5

. 2015 Apr 7;2015(4):CD002892. doi: 10.1002/14651858.CD002892.pub5

Preventing occupational stress in healthcare workers

Jani H Ruotsalainen ^1,^✉, Jos H Verbeek ¹, Albert Mariné ², Consol Serra ^3,^4,⁵

Editor: Cochrane Work Group

PMCID: PMC6718215 PMID: 25847433

Abstract

Background

Healthcare workers can suffer from occupational stress as a result of lack of skills, organisational factors, and low social support at work. This may lead to distress, burnout and psychosomatic problems, and deterioration in quality of life and service provision.

Objectives

To evaluate the effectiveness of work‐ and person‐directed interventions compared to no intervention or alternative interventions in preventing stress at work in healthcare workers.

Search methods

We searched the Cochrane Central Register of Controlled Trials (CENTRAL), MEDLINE, EMBASE, PsycINFO, CINAHL, NIOSHTIC‐2 and Web of Science up to November 2013.

Selection criteria

Randomised controlled trials (RCTs) of interventions aimed at preventing psychological stress in healthcare workers. For organisational interventions, interrupted time‐series and controlled before‐and‐after (CBA) studies were also eligible.

Data collection and analysis

Two review authors independently extracted data and assessed trial quality. We used Standardised Mean Differences (SMDs) where authors of trials used different scales to measure stress or burnout. We combined studies that were similar in meta‐analyses. We used the GRADE system to rate the quality of the evidence.

Main results

In this update, we added 39 studies, making a total of 58 studies (54 RCTs and four CBA studies), with 7188 participants. We categorised interventions as cognitive‐behavioural training (CBT) (n = 14), mental and physical relaxation (n = 21), combined CBT and relaxation (n = 6) and organisational interventions (n = 20). Follow‐up was less than one month in 24 studies, one to six in 22 studies and more than six months in 12 studies. We categorised outcomes as stress, anxiety or general health.

There was low‐quality evidence that CBT with or without relaxation was no more effective in reducing stress symptoms than no intervention at one month follow‐up in six studies (SMD ‐0.27 (95% Confidence Interval (CI) ‐0.66 to 0.13; 332 participants). But at one to six months follow‐up in seven studies (SMD ‐0.38, 95% CI ‐0.59 to ‐0.16; 549 participants, 13% relative risk reduction), and at more than six months follow‐up in two studies (SMD ‐1.04, 95% CI ‐1.37 to ‐0.70; 157 participants) CBT with or without relaxation reduced stress more than no intervention.

CBT interventions did not lead to a considerably greater effect than an alternative intervention, in three studies.

Physical relaxation (e.g. massage) was more effective in reducing stress than no intervention at one month follow‐up in four studies (SMD ‐0.48, 95% CI ‐0.89 to ‐0.08; 97 participants) and at one to six months follow‐up in six studies (SMD ‐0.47; 95% CI ‐0.70 to ‐0.24; 316 participants). Two studies did not find a considerable difference in stress between massage and taking extra breaks.

Mental relaxation (e.g. meditation) led to similar stress symptom levels as no intervention at one to six months follow‐up in six studies (SMD ‐0.50, 95% CI ‐1.15 to 0.15; 205 participants) but to less stress in one study at more than six months follow‐up. One study showed that mental relaxation reduced stress more effectively than attending a course on theory analysis and another that it was more effective than just relaxing in a chair.

Organisational interventions consisted of changes in working conditions, organising support, changing care, increasing communication skills and changing work schedules. Changing work schedules (from continuous to having weekend breaks and from a four‐week to a two‐week schedule) reduced stress with SMD ‐0.55 (95% CI ‐0.84 to ‐0.25; 2 trials, 180 participants). Other organisational interventions were not more effective than no intervention or an alternative intervention.

We graded the quality of the evidence for all but one comparison as low. For CBT this was due to the possibility of publication bias, and for the other comparisons to a lack of precision and risk of bias. Only for relaxation versus no intervention was the evidence of moderate quality.

Authors' conclusions

There is low‐quality evidence that CBT and mental and physical relaxation reduce stress more than no intervention but not more than alternative interventions. There is also low‐quality evidence that changing work schedules may lead to a reduction of stress. Other organisational interventions have no effect on stress levels. More randomised controlled trials are needed with at least 120 participants that compare the intervention to a placebo‐like intervention. Organisational interventions need better focus on reduction of specific stressors.

Plain language summary

Preventing occupational stress in healthcare workers

Background

Healthcare workers suffer from work‐related or occupational stress. Often this is because healthcare workers face high expectations and they may not have enough time, skills and social support at work. This can lead to severe distress, burnout or physical illness. In the end, healthcare workers may be unable to provide high quality healthcare services. Stress and burnout can also be costly because affected healthcare workers take sick leave and may even change jobs.

We evaluated how well different ways to prevent healthcare workers' stress or burnout work.

Study characteristics

We included 58 studies that included altogether 7188 participants. Fiftyfour of the included studies were randomised controlled studies and four were non‐randomised studies. We categorised the interventions as either cognitive‐behavioural training, mental and physical relaxation, or organisational changes.

Key findings and quality of the evidence

Cognitive‐behavioural interventions

According to six studies, there was low‐quality evidence that cognitive‐behavioural training decreased stress with about 13% when compared to no intervention and when measured at follow‐up periods ranging from less than a month up to two years. It is unclear how relevant this reduction is for a person with stress. The results were similar when cognitive‐behavioural training was combined with relaxation. However, in three studies, stress levels were similar after a cognitive‐behavioural training course compared to other training that was not focused on stress management but on the content of care.

Mental and physical relaxation interventions

In 17 studies there was low‐ to moderate‐quality evidence that both mental and physical relaxation led to a reduction of 23% in stress levels compared to no intervention.

Organisational interventions

Organisational interventions were aimed at changing working conditions in 20 studies, improving support or mentoring in six studies, changing content of care in four studies, improving communication skills in one study and improving work schedules in two studies. Shorter or interrupted work schedules reduced stress levels in two studies but there was no clear benefit of any of the other organisational interventions.

Conclusions

We conclude that cognitive‐behavioural training as well as mental and physical relaxation all reduce stress moderately. Changing work schedules can also reduce stress, but other organisational interventions have no clear effects. We need randomised studies with at least 120 participants and preferably a single component intervention. Organisational interventions need to be better focused on addressing specific factors that cause stress.

Summary of findings

Background

Healthcare workers can suffer from work‐related or occupational stress as a result of organisational factors and an imbalance of demands, skills and social support at work, or any combination of these. Under some conditions this can lead to severe distress, burnout or psychosomatic diseases, and the resulting deterioration in quality of life and service provision (Weinberg 2000).

Description of the condition

'Burnout' has been defined as a persistent, negative, work‐related state of mind in 'normal' individuals that is primarily characterised by exhaustion, which is accompanied by distress, a sense of reduced effectiveness, decreased motivation, and the development of dysfunctional attitudes and behaviours at work. This psychological condition develops gradually but may remain unnoticed for a long time for the individual involved. It results from a misfit between intentions and reality at the job. Often burnout is self‐perpetuating because of inadequate coping strategies that are associated with the syndrome (Schaufeli 2003). Burnout and occupational stress are both considered to result from exposure to stressors at work, the effects of which are mediated by personal coping or the ability to deal with environmental stressors at a personal level (Cooper 2001; Schaufeli 2003). Burnout is considered here as a specific form of psychological stress and not as a clinical diagnosis (Firth‐Cozens 1999). The economic impact of such conditions is high, as can be inferred from data on absenteeism and turnover (Jacobson 1996; Raiger 2005). More than 10% of total claims for occupational diseases are attributed to stress at work (Williamson 1994). There are a variety of stress factors in the workplace of healthcare workers that have been shown to increase the risk of distress and burnout, such as increasing workload, emotional response to contact with suffering and dying patients, and organisational problems and conflicts (Payne 1987; McNeely 2005). In addition, many studies have shown that levels of dissatisfaction, distress and burnout at work are quite high in healthcare workers (Deckard 1994; Karasek 1992; Maslach 1982; Raiger 2005; Ramirez 1996).

Description of the intervention

Work‐related stress and burnout can be addressed with one or more of the following approaches: teaching cognitive‐behavioural techniques, engaging in mental or physical relaxation or by changing the organisation of work.

How the intervention might work

Cognitive‐behavioural techniques work by providing new ways to feel, think and act in stressful situations. Mental and physical relaxation techniques divert attention away from unpleasant stressful thoughts and feelings and build resilience. Organisational interventions can prevent stressful events from occurring or the feeling of stress or burnout from developing by adjusting work practices so that they match and make use of workers' capabilities better.

Why it is important to do this review

An extensive number of reviews have been published on the effectiveness of interventions to prevent or treat stress (DeFrank 1987; Lamontagne 2007; Murphy 1995; Van der Hek 1997; Van der Klink 2001). However, apart from Jones 2000b and Mimura 2003, there are no other reviews that focus specifically on interventions directed at stress prevention in healthcare workers. According to the Fifth European Working Conditions Survey (Eurofound 2012), the healthcare sector rates the highest on two important potential causes of work stress: hiding one's own emotions and adverse social behaviour, with 'mental health at risk' at an intermediate level compared with other sectors. In the UK, the healthcare sector has the highest estimated prevalence rate of work‐related stress (HSE 2013).

Because the characteristics of interventions designed for healthcare workers may be different from those of other occupations, the aim of this review is to determine the effectiveness of interventions to reduce stress specifically in healthcare workers. This is a fully updated version of the review that we first published in 2006.

Objectives

To evaluate the effectiveness of work‐ and person‐directed interventions compared to no intervention or alternative interventions in preventing stress at work in healthcare workers.

Methods

Criteria for considering studies for this review

Types of studies

For interventions directed at persons or the person/work interface such as relaxation or cognitive‐behavioural interventions, we considered only randomised controlled trials.

For work‐directed interventions such as organisational changes (see Types of interventions) it is usually unfeasible to randomise study participants because the interventions are usually executed at the environmental or group level. This can be overcome by using the cluster‐randomised design where workplaces or departments are randomised to the intervention or control groups. However, as this is a difficult and fairly new design, we also considered additional study designs for inclusion that allow for environmental interventions at the group level: controlled before‐and‐after studies (otherwise known as prospective cohort studies or quasi‐experimental studies) and interrupted time‐series. The interrupted time‐series design is defined by the Cochrane Effective Practice and Organisation of Care Review Group as a series of at least three outcome measurements before and at least three after the implementation of an intervention (EPOC 2013).

Types of participants

We included studies in which the interventions were directed at workers who had not actively sought help for conditions such as burnout, depression or anxiety disorder. This included interventions directed at healthcare workers officially employed in any healthcare setting or at student nurses or physicians otherwise in training to become a professional who were also doing clinical work. This excluded studies in which the participants were simply caregivers and were not employed by a healthcare organisation.

Types of interventions

We included studies with any kind of intervention aimed at preventing or reducing stress arising from work. 'Prevention' is defined as an intervention directed at healthy individuals not diagnosed with a specific disease.

We categorised interventions into three groups:

1. Cognitive‐behavioural interventions that aim to change the way in which participants think, feel and consequently behave in stressful situations.

2. Relaxation interventions that aim to induce a state of mental or bodily calmness, or both, to counteract the agitation caused by stress. This can be achieved by, for example, being a passive recipient of a massage or by actively performing various exercises. Thus focus is directed towards a specific relaxing activity and away from the unpleasant thoughts and feelings associated with stress.

3. Organisational interventions that are measures to change resources, the working environment, work tasks or working methods. The intention here is to prevent or decrease stress by ensuring that workers are better equipped to deal with the tasks associated with their jobs.

Some authors also distinguish multimodal interventions that consist of cognitive‐behavioural interventions combined with, for example, relaxation. We considered it more informative to describe such studies as combining cognitive‐behavioural interventions with relaxation.

For all kinds of interventions we included studies with both non‐intervention controls and with one or more alternative interventions as a control group.

In the previous version of this review (Marine 2006) we categorised interventions as person‐ and work‐directed. We think the new categorisation is more informative.

Types of outcome measures

Primary outcomes

As the primary outcome we considered all validated self‐report questionnaires measuring occupational stress or burnout. These included measures such as: Maslach Burnout Inventory (MBI) (Maslach 1982), the Nursing Stress Scale (Gray‐Toft 1981) and Karasek's Job Content Questionnaire (Karasek 1998).

Secondary outcomes

As secondary outcomes we considered all outcome measures of the detrimental effects of stress or burnout. These included measures such as:  (a) Psychological symptoms: anxiety and depression, such as the State‐Trait Anxiety Inventory (Spielberger 1970), General Health Questionnaire (Goldberg 1991), Beck Depression Inventory (Beck 1961);   (b) Physical symptoms and physiological parameters: for example, hormone levels such as prolactin, corticosteroids or others;  (c) Measures on the cost effectiveness of interventions, such as incremental cost‐effectiveness ratios (ICERs), incremental cost‐per‐QALY (quality‐adjusted life year) and cost‐benefit ratios.

We considered the effects on the following time‐scales:  (i) Up to one month  (ii) From one month to six months  (iii) Over six months

We deemed all other outcomes that do not measure stress or its effects on individuals beyond the scope of this review and thus we excluded them from analyses. Examples of excluded outcomes are: coping skills, knowledge or attitude change, work performance, patient satisfaction and claims from clients, employee absenteeism and turnover.

Search methods for identification of studies

Electronic searches

The original systematic searches:  (a) MEDLINE/PubMed (1966 ‐ May 2005)  (b) PsycINFO/Ovid (1967 ‐ May 2005)  (c) Cochrane Depression, Anxiety and Neurosis Group (CCDAN) Specialised registry (last search: February 2002)  (d) Cochrane Occupational Health Field (COHF) Specialised registry (last search: May 2005)    The updated systematic searches:

(a) Cochrane Central Register of Controlled Trials (CENTRAL) (The Cochrane Library, 2013, Issue 11 (November))  (b) MEDLINE/PubMed (1966 ‐ November 2013)  (c) EMBASE.COM (1986 ‐ November 2013)  (d) PsycINFO/ProQuest (1967 ‐ November 2013)  (e) CINAHL/EBSCO (1981 ‐ November 2013)  (f) NIOSHTIC‐2 (1900 ‐ December 2013)  (g) Web of Science (1988 ‐ November 2013)

Searching other resources

Reference lists  We carefully examined the reference lists from articles and reviews for any additional eligible studies.

Handsearching   We handsearched all issues of Work & Stress between January 1987 and November 2013.

Data collection and analysis

Selection of studies

Two review authors (JR, JV) independently selected the studies to include in the review according to the inclusion criteria. If there was any disagreement concerning the inclusion of a study, we discussed this jointly, and where necessary a third review author (AM or CS) resolved the disagreement.

Data extraction and management

We conducted the extraction of data by using a made‐to‐measure data extraction form filled in independently by two review authors (JR and JV or AM and CS) for each study.  (a) If studies used a cross‐over design we used the results from just after the implementation of the intervention in the intervention group, compared to the results in the concurrent waiting‐list control group.  (b) If studies compared more than one active intervention we chose the most intensive intervention to enter into the meta‐analysis.

Assessment of risk of bias in included studies

We used the following items to assess risk of bias in the included studies: adequate sequence generation, allocation concealment, incomplete outcome data addressed and selective outcome reporting. We did not use items on blinding because none of the studies could blind participants or intervention providers, and all primary outcomes were from self‐reported questionnaires.

Measures of treatment effect

We plotted the results of each trial as means and standard deviations (SDs) for continuous outcomes. Because in many cases different instruments were used to measure stress, we transformed the means into standardised mean differences (SMDs) or effect sizes. Because these outcomes are difficult to interpret for clinical use, we also reported the means and standard deviations for the most‐used stress‐measurement instrument and the MBI.

In many cases multiple similar outcome measures were used or an instrument had several sub‐scales but no summary measure. In case of multiple similar outcomes, we chose the outcome which we deemed to best represent a measure of stress in healthcare workers, such as the Nursing Stress Scale (Gray‐Toft 1981) instead of, for example, the Perceived Stress Scale (Cohen 1983). When study authors used subscales such as with the MBI (Maslach 1996), we chose the subscale that in our view best represented stress, such as the emotional exhaustion scale of the MBI (Maslach 1996) or the effort subscale of the Effort Reward Imbalance Questionnaire (Siegrist 2004).

Unit of analysis issues

For studies that employed a cluster‐randomised design and that reported sufficient data to be included in the meta‐analysis and that did not make an allowance for the design effect, we calculated the design effect based on a fairly large assumed intra‐cluster correlation of 0.10. Even though we did not find information for the intra‐cluster correlation for these types of studies we assumed that 0.10 would be a realistic estimate. We based this assumption by analogy on studies about implementation research (Campbell 2001). We followed the methods stated in the Cochrane Handbook for Systematic Reviews of Interventions (Cochrane Handbook, Higgins 2011) for the calculations: design effect = 1+(M‐1)*ICC, where M is the average cluster size and ICC is the intra‐cluster correlation coefficient.

For studies with multiple study‐arms, we decided which was the least extensive intervention and compared all other arms to that intervention only, to prevent increasing the number of comparisons. When more than one arm of an intervention was included in the same comparison, we divided the number of control participants in two and used half of the control participants for each intervention arm to prevent double‐counting.

For studies with a cross‐over design we judged if a sufficient wash‐out period was used in the study design. We expected that the effects of most stress‐management interventions would extend over longer periods of time, and for these types of studies we only used data from before the washout period. For studies where a more immediate effect would be expected, such as in studies that evaluated working schedules, we accepted that with a reasonable washout period the results of the second period could also be used.

Dealing with missing data

Where necessary, we sought missing statistical data (means and standard deviations) from authors. We received unpublished statistical data from the authors of the following studies: Cohen‐Katz 2005; Ketelaar 2013; Gärtner 2013; Jones 2000a; McElligott 2003; Moody 2013; Oman 2006. The first author of Bittman 2003 did not wish to share their data and the first author of Ewers 2002 did not have the data anymore. The authors of Razavi 1993 referred us to the results of Delvaux 2004 instead. The author of Rowe 2006 reported that after baseline they had only measured MBI, which explains the apparent selective reporting.

When standard deviations (SDs) were not reported we calculated them from other reported values according to the methods stated in the Cochrane Handbook (Higgins 2011). For West 1984 we took the means and SDs that resulted from the post hoc comparisons in the repeated measures analyses. For Norvell 1987, we took the post‐treatment values and calculated SDs based on the P value. We calculated a t‐value from this P value even though the authors used a Mann‐Whitney U test. For Shapiro 2005, we took the post‐treatment values and the F‐value reported by the authors. We calculated a t‐value and subsequent SDs by taking the square root of the F‐value as the t‐value. For Tsai 1993, we took the post‐treatment values from the figure reporting the results of the repeated measures analysis. We took the reported P value belonging to the repeated measures analysis as if it had resulted from a t‐test and calculated the SDs based on this t‐value. For Ewers 2002, we took the post‐treatment scores and the P values belonging to the independent t‐tests to calculate a t‐value and subsequently SDs. For Martins 2011, we imputed the average of the SDs from the other studies in the same comparison.

Assessment of heterogeneity

We assessed first if studies were clinically homogeneous, i.e. sufficiently similar regarding their participants, interventions, control condition, outcome and follow‐up time to be combined in one comparison. Then we assessed statistical heterogeneity with the I² statistic. We deemed an I² value of more than 50% to indicate considerable heterogeneity.

Assessment of reporting biases

We avoided reporting bias by including studies and not articles. If multiple articles reported reported results from a single study, we consolidated all the data from all articles under one study ID only. We avoided language bias by including studies in any language. The Cochrane Handbook (Higgins 2011) recommends assessing publication bias with funnel plots when there are at least ten studies in a comparison. However, as this recommended cut‐off is arbitrary, we assessed publication bias with funnel plots in comparisons containing five or more studies.

Data synthesis

We combined studies that we deemed sufficiently similar regarding participants, intervention, control, outcome and follow‐up time in one comparison.

We pooled the results statistically when the outcomes were similar concepts, such as perceived stress. Because many different instruments were used, we used standardised mean differences (SMDs) to combine the stress‐related outcomes in the meta‐analysis.Not all instruments used one summary score but presented the results of various subscales. In cases where there was no summary measure, we chose the subscale that best represented a measure of stress. For example, for this analysis, we used only the emotional exhaustion subscale of the Maslach Burnout Inventory. In this way, we considered the various stress scales to measure the same concept.

To better enable the interpretation of these results we recalculated the pooled SMD back into a mean difference for the emotional exhaustion subscale of the MBI by dividing the SMD by an estimation of standard deviation. For this estimation we used the median value of the SDs of the control group in the studies that evaluated this intervention and used the MBI.

If there was little or no statistical heterogeneity in a comparison, we pooled the results using a fixed‐effect model. If the I² statistic was more than 50% we combined the study results using a random‐effects model.

Subgroup analysis and investigation of heterogeneity

Since various occupational groups of healthcare workers have different work content, their stress levels could be different and also the effect of interventions. We therefore evaluated the effect in nurses, physicians, and other health professionals like physiotherapists in subgroups if available in the same comparison.

When substantial heterogeneity was present in a comparison as indicated by I² greater than 50%, we tried to find an explanation. In our opinion, the most likely explanation is that there are data input errors.

Sensitivity analysis

To assess the effect of risk of bias on the pooled results, we performed a sensitivity analysis in which we excluded studies with a high risk of bias and assessed whether this changed the results appreciably.

Results

Description of studies

Results of the search

From the initial set of references found with the original systematic searches conducted in 2005, we identified a set of 51 potentially eligible studies (Figure 1). We scrutinised these further, and excluded 31 studies, leaving a total of 20 studies. However, we noticed that two studies actually reported separate sets of results of a single study so we included them as one study and henceforth refer to it as Lökk 2000. Thus, the original sample consisted of 19 studies. The first updated systematic searches conducted in 2008 yielded a set of 8623 references, after excluding duplicates, of which 5334 covered the period of the original search. Screening these references for eligibility resulted in 17 of the 19 studies already included in the original sample and an additional 30 potentially eligible studies for the period covered by the original search. This showed that the original search had not been sensitive enough. The first updated search in 2008 yielded another 34 potential studies that had been published since 2005. Other sources resulted in another six potential studies. We scrutinised the full‐text articles of these 70 new potentially eligible studies with regard to our inclusion and exclusion criteria. Altogether we excluded 56 studies, leaving a total of 14 new studies. However, one of these new studies (Rowe 2006) actually reported new follow‐up data for an earlier study (Rowe 1999). Because the more recent article reported the results of all measurements starting from baseline, we used these data and thus included the study only once. The systematic searches have since been updated three times, in November 2010, May 2012 and November 2013. These yielded altogether 9646 references, excluding duplicates. We assessed 100 full text articles for eligibility and excluded 76. This left 24 new studies. Put together, via four rounds of systematic searches conducted in 2008, 2010, 2012 and 2013, we have identified altogether 58 studies that fulfilled our inclusion criteria and 212 that did not and that we summarily excluded. In addition, we located three ongoing studies (Gomez‐Gascon 2013; Niks 2013; Spoor 2010) for which we could not find published outcome data.

We sought additional information regarding study details and statistical data or both from 18 authors and we received it from 12. Eight of the authors provided data that had not been published in their articles which enabled us to enter two of these studies into the meta‐analyses.

Included studies

Study designs   Of the 58 included studies, 42 were randomised controlled trials, eight were cluster‐randomised trials, four were cross‐over studies and four were controlled before‐and‐after studies of a work‐directed intervention.

Six of the eight included cluster‐randomised trials had a unit of analysis error. In other words, these studies ignored the clustering of the data in their analysis. Hence we had to adjust the data accordingly. See Unit of analysis issues for the formula we used to calculate the design effect based on average cluster size (M) and an intra‐cluster correlation coefficient. We calculated the design effect as 2.01 for the Jensen 2006 study (19 clusters, M = 11.05); 2.2 for the Lökk 2000 study (2 clusters, M = 13); 2.73 for the Schrijnemaekers 2003 study (16 clusters, M = 18.3); 1.72 for the Proctor 1998 study (12 clusters, M = 8.2), 2.7 for Uchiyama 2013 (24 clusters, M = 18.08) and 1.42 for Ketelaar 2013 (58 clusters, M = 5.17). We used the design effect to reduce the number of participants in both intervention and control groups if we were able to use quantitative outcome data in meta‐analyses.

For the four cross‐over studies we wanted to use data from the first period only for Bittman 2003 and Lai 2011, but the studies did not provide these and thus could not contribute to the analysis. Von Baeyer 1983 reported the results of the first period separately and we included these. Lucas 2012 studied the immediate effect of different working schedules and used a washout period of four weeks between the intervention and the control period, which seemed reasonable. For this study we included the post‐intervention results of all participants.

We were aiming also to include studies with interrupted time‐series designs but the final sample contained none of these.

Country and time period   Twelve of the included studies had been conducted before the year 2000, 22 between the years 2000 and 2010, and 24 in 2010 or later. Nineteen studies had been carried out in Europe, another 24 in North America, eight in Asia, three in the Middle East, two in South America and two in Australia.

Type of settings and participants   Altogether 39 of the included studies had been conducted in hospitals, eight in residential care homes for the elderly or the disabled, seven in mixed healthcare settings, three in a secure unit and one in a nursing school. Thirty‐five studies included exclusively nurses (including all institutions), 15 included all healthcare staff, but usually the majority of the staff were nurses and only a small fraction (about 5%) were physicians, six studies exclusively included physicians and two included either physiotherapists or respiratory therapists.

Sample sizes   The total number of participants was 7188, with 3592 in various intervention groups and 3596 in control groups. The number of participants was less than 20 in five studies, more than 20 but less than 60 in 27 studies, more than 60 but less than 300 in 20 studies, and more than 300 in six studies.

Interventions

Thirteen study‐arms (Delvaux 2004; Gardner CBT 2005; Gardner Coping 2005; Ketelaar 2013; Gärtner 2013; Günüsen 2010; Jensen 2006; Kwok 2012; Lee 1994; Lökk 2000; Martins 2011; Rowe 2006; Yamagishi 2008) examined cognitive‐behavioural interventions.

Six studies (Jones 2000a; Norvell 1987; Reynolds 1993; Sood 2011; Von Baeyer 1983; West 1984) assessed a combination of cognitive‐behavioural elements and relaxation.

Twenty‐two study‐arms examined relaxation interventions. We categorised relaxation interventions as: physical: Bittman 2003; Brennan 2006; Griffith 2008; Hansen 2006; Kurebayashi 2012; Lai 2011; Lemaire 2011; McElligott 2003; Palumbo 2012; Saganha 2012; Tsai 1993; Yazdani 2010; Yung 2004; and mental relaxation: Cohen‐Katz 2005; Klatt 2012; Mackenzie 2006; Moody 2013; Oman 2006; Shapiro 2005; Stanton 1988.

Twenty‐one study‐arms examined the effectiveness of organisational interventions. Of these, six tried to improve working conditions with or without the participation of workers (Bourbonnais 2011; Le Blanc 2007; Melchior 1996; Proctor 1998; Romig 2012; Uchiyama 2013). Another seven organised some kind of support, for example peer support groups, to discuss problems at work that increase stress (Carson 1999; Günüsen 2010; Heaney 1995; Leiter 2011; Li 2011; Peterson 2008; Salles 2013). Four studies evaluated the introduction of special care models such as emotion‐oriented care training (Finnema 2005; Razavi 1993; Redhead 2011; Schrijnemaekers 2003). Two studies evaluated the effect of different work schedules on stress (Ali 2011; Lucas 2012), one assessed the effect of improving nurses' knowledge about serious mental illness on stress (Ewers 2002) and another evaluated the effect of communication skills training on stress (Ghazavi 2010).

Type of control group (no intervention versus other active or 'passive' interventions)    Almost all (45) of the included studies used a no‐intervention or waiting‐list control group.

Eleven studies compared an active intervention to a 'passive' intervention aiming to control for the Hawthorne effect, etc. (Brennan 2006; Carson 1999; Finnema 2005; Jensen 2006; Lai 2011; Lee 1994; Lökk 2000; Moyle 2013; McElligott 2003; Salles 2013; Tsai 1993).

Two studies (Ali 2011; Lucas 2012) compared only active interventions, that is, different work schedules with one another.

Multiple Intervention Arms

Seven studies (Gardner CBT 2005; Gärtner 2013; Günüsen 2010; Kurebayashi 2012; Rowe 2006; West 1984; Yung 2004) compared one or more active stress management interventions. Gardner CBT 2005 included an arm with CBT content and an arm aimed at improving coping with stress with relaxation and other individual self‐management interventions (time management, assertion, problem‐solving, etc.). We used both arms in different subgroups. Ketelaar 2013 and Gärtner 2013 evaluated two interventions with a similar content but with different modes of providing the intervention. We therefore combined those in the same comparison and again halved the numbers in the control group. Günüsen 2010 provided a coping and work support intervention and these were entered in separate comparisons. With Kurebayashi 2012, we entered both interventions in the same comparison of physical relaxation. West 1984 had five study arms but finally reported data only on one study arm versus a no‐intervention or no‐effect condition. We used this as an intervention versus no‐intervention comparison.

Outcomes

Altogether 21 studies used the Maslach Burnout Inventory (MBI) or one of its subscales as their main outcome measure. However, the MBI was only reported as a sum score in Shapiro 2005, and we used the Perceived Stress Scale (PSS) for that study instead of the MBI. Six studies used the Perceived Stress Scale (Brennan 2006; Griffith 2008; Lee 1994; Li 2011; Palumbo 2012; Sood 2011). Four studies used their own definition of burnout (Ali 2011; Bourbonnais 2011; Romig 2012; Salles 2013). Two studies used the Nurse Stress Scale (Delvaux 2004; Razavi 1993). Two studies used the General Health Questionnaire (GHQ) (Reynolds 1993; Yung 2004) and another two used the Profile of Moods States (POMS) (Bittman 2003; Moyle 2013). Two studies used their own developed stress scale (Ghazavi 2010; Lemaire 2011). The other outcome measures were each used by one study only: 4‐dimension complaint checklist (Ketelaar 2013), Brief Job Stress Questionaire (Yamagishi 2008), Depression Anxiety Stress Scale (Yazdani 2010), Derogatis Stress Profile (Jones 2000a), Effort Reward Imbalance Questionnaire (Uchiyama 2013), Job Stress Questionnaire (Hansen 2006), The Mental Health Professional Stress Scale (Gardner CBT 2005), Nurse Stress Checklist (see Benoliel 1990) (Tsai 1993), Occupational Stress Indicator (see Cooper 1988) (Proctor 1998), Organisation and Stress Scale (Finnema 2005), The General Nordic Questionnaire for Psychological and Social Factors at Work (QPS Nordic) (Peterson 2008), Symptom Checklist‐90 (see Derogatis 1976; Derogatis 1983) (Heaney 1995), Somatic Stress (Kwok 2012), Stress Profile (Stanton 1988), Stress Symptom Checklist (Kurebayashi 2012), Structured Stress Questionnaire (Lökk 2000), and Stress Visual Analogue Scale (Lai 2011). Two studies measured only anxiety (McElligott 2003; Von Baeyer 1983).

Altogether 22 studies used more than one of the stress measures listed above.

None of the studies measured the cost effectiveness of interventions.

Follow‐up  (i)Short‐Term:   There were 24 studies in which the last outcome measurement was either at the end of the intervention (Ewers 2002; Finnema 2005; Ghazavi 2010; Griffith 2008; Hansen 2006; Kurebayashi 2012; Lai 2011; Lemaire 2011; Li 2011; Lucas 2012; Mackenzie 2006; Martins 2011; McElligott 2003; Moyle 2013; Norvell 1987; Palumbo 2012; Proctor 1998; Saganha 2012; Shapiro 2005; Sood 2011; Von Baeyer 1983; Yamagishi 2008) or at less than a month after intervention (Brennan 2006; Tsai 1993).

(ii) Medium‐Term:   In another 22 studies the last follow‐up measurement was between one and six months after intervention (Bittman 2003; Carson 1999; Cohen‐Katz 2005; Delvaux 2004; Gardner CBT 2005; Günüsen 2010; Heaney 1995; Klatt 2012; Kwok 2012; Le Blanc 2007; Lee 1994; Leiter 2011; Lökk 2000; Moody 2013; Oman 2006; Razavi 1993; Reynolds 1993; Romig 2012; Salles 2013; West 1984; Yazdani 2010; Yung 2004).

(iii) Long Term:   There were also 12 studies in which the last follow‐up measurement was at six months after intervention or later (Ali 2011; Bourbonnais 2011; Jensen 2006; Jones 2000a; Ketelaar 2013; Melchior 1996; Peterson 2008; Redhead 2011; Rowe 2006; Schrijnemaekers 2003; Stanton 1988; Uchiyama 2013).

Excluded studies

Excluded studies

The main reasons for excluding studies from this review were the following, as illustrated by selected examples of each category (see Characteristics of excluded studies):

No control group: Andersen 2010; Cohen 2005; Elo 2000; Michie 1992; Petterson 2006; Schaufeli 1995; Yamagishi 2007.
Non‐randomised study design and intervention directed at persons or the person/work interface: Bourbonnais 2006a; Holt 2006; Michie 1994; Petterson 1998; Young 2001; Zimber 2001.
No intervention aimed at preventing or treating stress or burnout: Bourbonnais 2006b; Brinkborg 2011; Cámara Conde 2009; Ridge 2011; Wetzel 2011.
The intervention is primarily aimed at people with dementia at a care home and not the staff caring for them, although the effects were measured also for the latter: Baldelli 2004.
No primary outcome measure of stress or burnout: Bay 2010; Lai 2012; Pipe 2009; Saadat 2012; Sharif 2013; Villani 2012.
Participants were not healthcare workers or nursing students with clinical duties: Drain 1991.
Participants were nursing students but did not have clinical duties: Johansson 1991; Russler 1991; Sharif 2004.
Participants were victims of assault and deemed not healthy as per inclusion criteria: Nhiwatiwa 2003.
Participants had been diagnosed with burnout syndrome: Diaz‐Rodriguez 2011a; Diaz‐Rodriguez 2011b.

Risk of bias in included studies

In general, most studies were of low methodological quality, with at least several items that we judged to put them at a high risk of bias (Figure 2). We judged only one study (Günüsen 2010) to be at low risk of bias in randomisation, allocation concealment, incomplete data, selective reporting or other risks of bias. Blinding was consistently problematic in all studies because self report was used to assess stress levels as the outcome and the providers could not be blinded to the intervention. We therefore did not use this item to make a distinction between studies at high or low risk of bias.

Risk of bias summary: review authors' judgements about each risk of bias item for each included study.

Allocation

Only 12 out of the 58 studies clearly described the method for generating random numbers, with the newer studies being better in this regard than the older studies. It is still surprising to note that most studies just describe the whole process as "We randomised participants". The large number of studies that did not report details prevented us from asking all authors to provide better information. Details of allocation concealment were lacking even more often, with only six studies giving an adequate description. Even though many other risks of bias are difficult to avoid in stress management studies, these items can clearly be improved for any study. The fact that a minority of studies reported sufficient detail and content to justify an assessment of low risk of bias means that it is feasible for all.

Blinding

We considered the reporting of stress by questionnaires as an outcome assessment that could be biased by knowledge of the intervention. We judged that it could be possible that a participant in the intervention group, knowing that they have gone through a six‐week course of stress management, would rate their stress symptoms more favourably than a person in the control group. This would create an overestimation of the effect of the intervention. Very few authors mentioned that blinding could be an issue, but said also that they thought that blinding is impossible here. We agree that this is not possible but that does not eliminate the risk of bias and we therefore rated all studies with self reports as being at high risk of bias.

Incomplete outcome data

The loss of participants was mostly low. However, 12 studies (Brennan 2006; Carson 1999; Finnema 2005; Ketelaar 2013; Gärtner 2013; Griffith 2008; Jensen 2006; McElligott 2003; Melchior 1996; Palumbo 2012; Peterson 2008; Tsai 1993) suffered a loss exceeding 20% of the initial sample. These were labelled as being at high risk of bias. Also one study (Bittman 2003) reported the reasons for participants not completing the study but not separately for the two groups. As the reasons listed were not entirely random (e.g. poor attendance and refusal to participate) we judged this study to be at high risk of attrition bias. In six studies (Jones 2000a; Lee 1994; Razavi 1993; Reynolds 1993; Rowe 2006; Shapiro 2005) results were analysed only for participants who provided complete data sets. We labelled these studies as being at high risk of bias. In two studies (Mackenzie 2006; Stanton 1988) it was unclear whether any participants dropped out and the studies were therefore labelled as being at unclear risk of bias.

Selective reporting

In the absence of a protocol, it is difficult to judge if outcomes are reported as planned. If the authors mentioned a protocol, we checked the protocol. If there was no mention of a protocol we judged if the methods and results sections reported the same outcomes. In most (41) of the studies all outcomes were reported as planned and consequently we judged them as being at low risk of bias. In one study (Carson 1999) two of the outcomes (Minnesota Job Satisfaction scale and the Cooper Coping Skills scale) were administered but their results were not reported. We judged this to be a sign of a high risk of bias. In two studies (Gardner CBT 2005; Jensen 2006) only significant differences were reported, which we also took to be a sign of a high risk of bias. In Finnema 2005 the results for nursing assistants consisted of covariance analyses that were not prespecified and because of this we judged the study to be at high risk of bias. In Rowe 2006 authors report only the results of the Maslach Burnout Inventory (MBI) at each follow‐up point, but when we made contact the author explained that this was the plan all along. We therefore assessed the study as being at low risk of bias.

Other potential sources of bias

There were several risks of bias that came up in addition to the risks mentioned above, such as an unclear washout period in a cross‐over trial (Lai 2011), very low compliance with the intervention (Ketelaar 2013) or a loosely validated outcome measure (Ghazavi 2010). If we found nothing else then we left the other potential source of bias as unclear in the 'Risk of bias' tool.

Effects of interventions

See: Table 1; Table 2

Summary of findings for the main comparison. Cognitive‐behavioural intervention compared to no Intervention for reducing stress.

Cognitive‐behavioural intervention compared to no Intervention for reducing stress
Patient or population: Healthcare workers  Settings: Health Care  Intervention: Cognitive‐behavioural intervention  Comparison: No Intervention (SMD)
Outcomes	*Illustrative comparative risks (95% CI)**		Relative effect  (95% CI)	No of Participants  (studies)	Quality of the evidence  (GRADE)	Comments
	Assumed risk	Corresponding risk
	No Intervention	Cognitive‐behavioural intervention
Stress 1 Month   Various Measurement Instruments  Follow‐up: 0 ‐ 1 month	The median stress level across all control groups across all follow‐up times was 24.33 points on the Emotional Exhaustion subscale of the Maslach Burnout Inventory¹	The mean stress level at 1 month in the intervention groups was 1.22 points lower (2.98 lower to 0.59 higher).	SMD ‐0.27 (‐0.66 to 0.13)	332  (6 studies)	⊕⊕⊝⊝  low²^,³	This meta‐analysis was back transformed to the EE subscale of the MBI by using the median SD of the EE scale across control groups.
Stress 1 ‐ 6 months   Various Instruments. Scale from: 0 to 2.  Follow‐up: 1 ‐ 6 months	The median stress level across all control groups across all follow‐up times was 24.33 points on the Emotional Exhaustion subscale of the Maslach Burnout Inventory¹	The mean stress at 1 ‐ 6 months in the intervention groups was 1.83 points lower (0.77 to 2.85 lower).	SMD ‐0.38 (‐0.59 to ‐0.16)	549  (8 studies)	⊕⊕⊝⊝ low²^,³	This meta‐analysis was back transformed to the EE subscale of the MBI by using the median SD of the EE scale across control groups.
Stress > 6 months   Various Instruments  Follow‐up: > 6 months	The median stress level across all control groups across all follow‐up times was 24.33 points on the Emotional Exhaustion subscale of the Maslach Burnout Inventory¹	The mean stress at more than 6 months in the intervention groups was 11.34 points lower (4.47 to 14.94 lower)	SMD ‐1.04 (‐1.37 to ‐0.70)	157  (2 studies)	⊕⊕⊝⊝ low²^,³	This meta‐analysis was back transformed to the EE subscale of the MBI by using the median SD of the EE scale across control groups.
The basis for the assumed risk* (e.g. the median control group risk across studies) is provided in footnotes. The corresponding risk (and its 95% confidence interval) is based on the assumed risk in the comparison group and the relative effect of the intervention (and its 95% CI).  CI: Confidence interval
GRADE Working Group grades of evidence  High quality: Further research is very unlikely to change our confidence in the estimate of effect.  Moderate quality: Further research is likely to have an important impact on our confidence in the estimate of effect and may change the estimate.  Low quality: Further research is very likely to have an important impact on our confidence in the estimate of effect and is likely to change the estimate.  Very low quality: We are very uncertain about the estimate.

Relaxation compared to no intervention (SMD) for
Patient or population: Healthcare workers  Settings: Health care  Intervention: Relaxation  Comparison: No intervention (SMD)
Outcomes	*Illustrative comparative risks (95% CI)**		Relative effect  (95% CI)	No of Participants  (studies)	Quality of the evidence  (GRADE)	Comments
	Assumed risk	Corresponding risk
	no intervention (SMD)	Relaxation
Stress 1 month	The median stress level across all control groups across all follow‐up times was 22.17 points on the Emotional Exhaustion subscale of the Maslach Burnout Inventory¹	The mean stress at 1 month in the intervention groups was 2.14 points lower (0.36 to 3.96 lower).	SMD ‐0.48 (‐0.89 to ‐0.08)	97  (4 studies)	⊕⊕⊝⊝ low	This meta‐analysis was back transformed to the EE subscale of the MBI by using the median SD of the EE scale across control groups.
Stress 1 ‐ 6 months	The median stress level across all control groups across all follow‐up times was 22.17 points on the Emotional Exhaustion subscale of the Maslach Burnout Inventory¹	The mean stress at 1 ‐ 6 months in the intervention groups was 4.84 points lower (2.37 to 6.92 lower).	SMD ‐0.49 (‐0.78 to ‐0.21)	521  (12 studies)	⊕⊕⊕⊝ moderate	This meta‐analysis was back transformed to the EE subscale of the MBI by using the median SD of the EE scale across control groups.
Stress > 6 months	The median stress level across all control groups across all follow‐up times was 22.17 points on the Emotional Exhaustion subscale of the Maslach Burnout Inventory¹	The mean stress at > 6 months in the intervention groups was 5.67 points lower (3.39 to 7.95 lower).	SMD ‐1.89 (‐2.65 to ‐1.13)	40  (1 study)	⊕⊕⊝⊝ low	This result was back transformed to the EE subscale of the MBI by using the median SD of the EE scale across control groups.
The basis for the assumed risk* (e.g. the median control group risk across studies) is provided in footnotes. The corresponding risk (and its 95% confidence interval) is based on the assumed risk in the comparison group and the relative effect of the intervention (and its 95% CI).  CI: Confidence interval;
GRADE Working Group grades of evidence  High quality: Further research is very unlikely to change our confidence in the estimate of effect.  Moderate quality: Further research is likely to have an important impact on our confidence in the estimate of effect and may change the estimate.  Low quality: Further research is very likely to have an important impact on our confidence in the estimate of effect and is likely to change the estimate.  Very low quality: We are very uncertain about the estimate.

Comparison	Risk of Bias	Inconsistency	Indirectness	Imprecision	Publication Bias	Level of Evidence
CBT vs no intervention 1 mo follow‐up	5 out of 6 studies high risk of bias: 1 level down	I² = 64%: no downgrading	No	332 participants No downgrading	Yes in funnel plot: 1 level down	Low Quality
CBT vs no intervention 1 ‐ 6 mo follow‐up	5 out of 8 studies high risk of bias: 1 level down	I² = 54%: no downgrading	No	549 participants No downgrading	Yes in funnel plot: 1 level down	Low Quality
CBT vs no intervention > 6 months follow‐up	2 studies high risk of bias: 1 level down	I² = 38%: no downgrading	No	157 participants 1 level down	Not apparent	Low Quality
Relaxation vs no intervention 1 mo follow‐up	3 out 4 studies high risk of bias: 1 level down	I² = 0%: no downgrading	No	97 participants 1 level down	Not apparent	Low Quality
Relaxation vs no intervention 1 ‐ 6 mo follow‐up	10 out of 13 studies high risk of bias: 1 level down	I² = 57%: no downgrading	No	521 participants no downgrading	Not apparent	Moderate Quality
Relaxation vs no intervention > 6 mo follow‐up	1 study: no downgrading	N/A	No	40 participants 2 levels down	Not apparent	Low Quality

Date	Event	Description
31 March 2015	Amended	We received feedback from Paul Landsbergis and Ulla Peterson saying we had reported results of two included studies wrong. With Bourbonnais 2011, the mistake was a typo in the effects of interventions text describing length of follow‐up being less than a month when it was in fact three years. We corrected the text accordingly. With Peterson 2008, we had erroneously labelled an outcome as having been the General Health Questionnaire when the authors had actually used the Short Form Health Survey (SF‐36). We had also mistakenly reported the result as favouring no intervention because we had understood it meaning an increase in symptoms when in fact it favoured the organisational intervention group by showing an increase in general health. We switched the forest plot labels around and corrected the text, thus fixing the problem.
31 March 2015	Feedback has been incorporated	We appended the comments sent by Paul Landsbergis and Ulla Peterson verbatim into the Feedback section and also explained there how we amended the review.
5 December 2014	Amended	Corrected a small error (18% to 13%) in the PLS.
9 July 2014	New citation required and conclusions have changed	We added 38 studies and reorganised the analyses to be more informative.
30 October 2013	New search has been performed	We updated the review and merged it with the review entitled Preventive staff‐support interventions for health workers by Brian van Wyk and Victoria Pillay‐van Wyk because of considerable overlap.
4 November 2008	Amended	Converted to new review format.

Outcome or subgroup title	No. of studies	No. of participants	Statistical method	Effect size
1 Any Stress‐related Outcome (follow‐up up to 1 month)	7	332	Std. Mean Difference (IV, Random, 95% CI)	‐0.27 [‐0.66, 0.13]
1.1 Cognitive‐behavioural intervention only vs. no intervention	4	248	Std. Mean Difference (IV, Random, 95% CI)	‐0.25 [‐0.60, 0.11]
1.2 Cognitive‐behavioural intervention and relaxation vs. no intervention	3	84	Std. Mean Difference (IV, Random, 95% CI)	‐0.45 [‐1.61, 0.70]
2 Any Stress‐related Outcome (follow‐up 1 ‐ 6 months)	8	549	Std. Mean Difference (IV, Random, 95% CI)	‐0.38 [‐0.59, ‐0.16]
2.1 Cognitive‐behavioural intervention only vs. no intervention	6	439	Std. Mean Difference (IV, Random, 95% CI)	‐0.28 [‐0.47, ‐0.09]
2.2 Cognitive‐behavioural intervention and relaxation vs. no intervention	2	110	Std. Mean Difference (IV, Random, 95% CI)	‐0.78 [‐1.38, ‐0.18]
3 Any Stress‐related Outcome (follow‐up more than 6 months)	2	157	Std. Mean Difference (IV, Fixed, 95% CI)	‐1.04 [‐1.37, ‐0.70]
3.1 Cognitive‐behavioural intervention with or without relaxation vs. no intervention	2	157	Std. Mean Difference (IV, Fixed, 95% CI)	‐1.04 [‐1.37, ‐0.70]
4 State‐Trait Anxiety Inventory (follow‐up up to 1 month)	3		Mean Difference (IV, Random, 95% CI)	Subtotals only
4.1 CBT and relaxation vs. no intervention State Anxiety	3	135	Mean Difference (IV, Random, 95% CI)	‐11.07 [‐18.39, ‐3.75]
4.2 CBT and relaxation vs. no intervention Trait Anxiety	3	135	Mean Difference (IV, Random, 95% CI)	‐8.36 [‐10.02, ‐6.70]
5 State‐Trait Anxiety Inventory (follow‐up 1 ‐ 6 months)	1		Mean Difference (IV, Fixed, 95% CI)	Totals not selected
5.1 CBT and relaxation vs. no intervention State Anxiety	1		Mean Difference (IV, Fixed, 95% CI)	0.0 [0.0, 0.0]
5.2 CBT and relaxation vs. no intervention Trait Anxiety	1		Mean Difference (IV, Fixed, 95% CI)	0.0 [0.0, 0.0]
6 General Health Questionnaire (follow‐up up to 1 month)	1		Mean Difference (IV, Fixed, 95% CI)	Totals not selected
6.1 CBT and relaxation vs. no intervention	1		Mean Difference (IV, Fixed, 95% CI)	0.0 [0.0, 0.0]

Outcome or subgroup title	No. of studies	No. of participants	Statistical method	Effect size
1 Any stress scale (follow‐up 1 ‐ 6 months)	2	83	Std. Mean Difference (IV, Random, 95% CI)	‐0.54 [‐1.16, 0.08]
2 Any stress scale (follow‐up more than 6 months)	1		Std. Mean Difference (IV, Random, 95% CI)	Totals not selected
3 Anxiety (follow‐up 1 ‐ 6 months)	1		Mean Difference (IV, Fixed, 95% CI)	Totals not selected
3.1 CBT vs. passive support by a psychologist	1		Mean Difference (IV, Fixed, 95% CI)	0.0 [0.0, 0.0]

Outcome or subgroup title	No. of studies	No. of participants	Statistical method	Effect size
1 Any stress outcome (follow‐up up to 1 month)	4	97	Std. Mean Difference (IV, Random, 95% CI)	‐0.48 [‐0.89, ‐0.08]
1.1 Physical relaxation (follow‐up up to 1 month)	4	97	Std. Mean Difference (IV, Random, 95% CI)	‐0.48 [‐0.89, ‐0.08]
2 Any stress outcome (follow‐up 1 ‐ 6 months)	12	521	Std. Mean Difference (IV, Random, 95% CI)	‐0.49 [‐0.78, ‐0.21]
2.1 Mental relaxation	6	205	Std. Mean Difference (IV, Random, 95% CI)	‐0.50 [‐1.15, 0.15]
2.2 Physical relaxation	6	316	Std. Mean Difference (IV, Random, 95% CI)	‐0.47 [‐0.70, ‐0.24]
3 Any stress outcome (follow‐up more than 6 months)	1		Std. Mean Difference (IV, Fixed, 95% CI)	Totals not selected
3.1 Mental relaxation	1		Std. Mean Difference (IV, Fixed, 95% CI)	0.0 [0.0, 0.0]
4 State Anxiety (follow‐up 1 ‐ 6 months)	1	66	Mean Difference (IV, Fixed, 95% CI)	‐7.79 [‐11.24, ‐4.34]
4.1 Mental relaxation vs. no intervention	1	33	Mean Difference (IV, Fixed, 95% CI)	‐8.74 [‐13.94, ‐3.54]
4.2 Physical relaxation vs. no intervention	1	33	Mean Difference (IV, Fixed, 95% CI)	‐7.04 [‐11.65, ‐2.43]
5 Trait Anxiety (follow‐up 1 ‐ 6 months)	1	66	Mean Difference (IV, Fixed, 95% CI)	‐1.09 [‐4.53, 2.36]
5.1 Mental relaxation vs. no intervention	1	33	Mean Difference (IV, Fixed, 95% CI)	‐2.41 [‐6.93, 2.11]
5.2 Physical relaxation vs. no intervention	1	33	Mean Difference (IV, Fixed, 95% CI)	0.75 [‐4.58, 6.08]
6 General Health (follow‐up 1 ‐ 6 months)	2		Mean Difference (IV, Fixed, 95% CI)	Subtotals only
6.1 Mental relaxation vs. no intervention	1	33	Mean Difference (IV, Fixed, 95% CI)	‐7.10 [‐11.39, ‐2.81]
6.2 Physical relaxation vs. no intervention	2	70	Mean Difference (IV, Fixed, 95% CI)	‐3.22 [‐6.53, 0.08]

Outcome or subgroup title	No. of studies	Statistical method	Effect size
1 Any stress outcome	4	Std. Mean Difference (IV, Fixed, 95% CI)	Totals not selected
1.1 Mental relaxation training vs. training on theory analysis (follow‐up 1 ‐ 6 months)	1	Std. Mean Difference (IV, Fixed, 95% CI)	0.0 [0.0, 0.0]
1.2 Mental relaxation vs. relaxing in a chair (follow‐up to 1 month)	1	Std. Mean Difference (IV, Fixed, 95% CI)	0.0 [0.0, 0.0]
1.3 Physical relaxation vs. break (follow‐up to 1 month)	2	Std. Mean Difference (IV, Fixed, 95% CI)	0.0 [0.0, 0.0]
2 Any anxiety outcome	1	Std. Mean Difference (IV, Random, 95% CI)	Totals not selected
2.1 AMMA touch therapy vs. touch therapy without intent (follow‐up to 1 month)	1	Std. Mean Difference (IV, Random, 95% CI)	0.0 [0.0, 0.0]
3 General Health Questionnaire	1	Std. Mean Difference (IV, Fixed, 95% CI)	Totals not selected
3.1 Physical relaxation training vs. training on theory analysis (follow‐up 1 ‐ 6 months)	1	Std. Mean Difference (IV, Fixed, 95% CI)	0.0 [0.0, 0.0]

Outcome or subgroup title	No. of studies	No. of participants	Statistical method	Effect size
1 Maslach Burnout Inventory (follow‐up up to 1 month)	2		Mean Difference (IV, Fixed, 95% CI)	Subtotals only
1.1 Emotional exhaustion	2	158	Mean Difference (IV, Fixed, 95% CI)	‐2.01 [‐3.34, ‐0.68]
1.2 Depersonalisation	2	158	Mean Difference (IV, Fixed, 95% CI)	‐1.35 [‐2.33, ‐0.36]
1.3 Personal accomplishment (lack of)	2	158	Mean Difference (IV, Fixed, 95% CI)	0.16 [‐1.21, 1.54]
2 Maslach Burnout Inventory (follow‐up 1 to 6 months)	2		Mean Difference (IV, Fixed, 95% CI)	Subtotals only
2.1 Emotional exhaustion	2	128	Mean Difference (IV, Fixed, 95% CI)	‐2.13 [‐3.77, ‐0.49]
2.2 Depersonalisation	2	142	Mean Difference (IV, Fixed, 95% CI)	‐0.47 [‐1.53, 0.59]
2.3 Personal accomplishment (lack of)	2	143	Mean Difference (IV, Fixed, 95% CI)	‐0.62 [‐1.83, 0.59]

Outcome or subgroup title	No. of studies	Statistical method	Effect size
1 Maslach Burnout Inventory (follow‐up more than 6 months)	1	Mean Difference (IV, Fixed, 95% CI)	Totals not selected
1.1 Emotional Exhaustion	1	Mean Difference (IV, Fixed, 95% CI)	0.0 [0.0, 0.0]
1.2 Depersonalisation	1	Mean Difference (IV, Fixed, 95% CI)	0.0 [0.0, 0.0]
1.3 Personal Accomplishment (lack of)	1	Mean Difference (IV, Fixed, 95% CI)	0.0 [0.0, 0.0]

Outcome or subgroup title	No. of studies	No. of participants	Statistical method	Effect size
1 Maslach Burnout Inventory (follow‐up 1 ‐ 6 months)	4		Mean Difference (IV, Random, 95% CI)	Subtotals only
1.1 Emotional Exhaustion	4	137	Mean Difference (IV, Random, 95% CI)	‐1.06 [‐5.98, 3.87]
1.2 Depersonalisation	4	137	Mean Difference (IV, Random, 95% CI)	‐1.03 [‐2.70, 0.64]
1.3 Personal Accomplishment (lack of)	4	137	Mean Difference (IV, Random, 95% CI)	3.96 [0.70, 7.21]

Methods	Cross‐over study, USA
Participants	112 randomly selected staff of a retirement community
Interventions	1) Experimental: Recreational music‐making: 6 consecutive weekly 1‐hour sessions of playing drums and percussion instruments with Clavinova accompaniment (typically to a familiar tune) and non‐verbal expression (by playing the drums) in direct response to questions that were intended to inspire deep thought, contemplation and mutual respect.  2) Control: No intervention
Outcomes	MBI, POMS
Notes
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	"All subjects were randomly assigned to Group A or Group B (mean 10.6 subjects/group) based upon the crossover design." (p.3)
Allocation concealment (selection bias)	Unclear risk	Not reported
Blinding (performance bias and detection bias)   All outcomes	High risk	Not possible
Incomplete outcome data (attrition bias)   All outcomes	High risk	Reasons for not completing the study reported but not separately for the 2 groups.
Selective reporting (reporting bias)	High risk	All outcomes reported but no report about cross‐over interim analysis or use of proper t‐tests
Other bias	Unclear risk	We did not find any indications of other sources of bias.

Methods	RCT, USA
Participants	50 employees of the Denver Veterans Affairs Medical Center
Interventions	1) Experimental: Qigong exercise: participants assigned to the exercise group attended a 1‐hour, noon qigong class twice weekly and were asked to practice independently for 30 minutes on non‐class days with the aid of a DVD demonstrating the instructor performing the exercises. Classes were taught by a senior apprentice in qigong with over 17 years’ experience in qigong. In addition, participants were provided a manual that outlined the acupuncture meridians and the rationale for the exercises. During class sessions, participants practised movements until they experienced a sensation of stretching or pressure in the targeted acupuncture points. The qigong movements were synchronised with specific breathing patterns, and participants were instructed to inhale and exhale fully at a comfortable rate, with a target respiratory rate of 6 breaths or fewer per minute.  2) Control: No intervention
Outcomes	Perceived Stress Scale
Notes
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	"Fifty (50) subjects, 39 females and 11 males, were matched for gender and then randomized (25 in each treatment group) through the use of computer‐generated numbers (SYSTAT12.0 for Windows, Cranes Software, 2007, Bangalore, India)." (p. 940)
Allocation concealment (selection bias)	Unclear risk	Not reported
Blinding (performance bias and detection bias)   All outcomes	High risk	Not possible, self‐report
Incomplete outcome data (attrition bias)   All outcomes	High risk	"There were no significant differences in numbers of withdrawals between study groups" (p.940) This statement seems counterfactual as the intervention group lost 36% and the control group lost 16% of their participants before follow‐up. In the intervention group 5 (25%) stopped attending and in the control group one participant started mindfulness training. ITT analysis was not used.
Selective reporting (reporting bias)	High risk	Authors only report change values.
Other bias	Unclear risk	We did not find any indications of other sources of bias.

Methods	RCT of individuals, USA
Participants	Personnell of a surgical intensive care unit of a large university hospital (n = 32); 69% nurses, 88% women, age average 44 yrs, experience 12 ± 10 yrs
Interventions	1) Experimental: A pragmatic low‐dose mindfullness‐based worksite intervention, on‐site 1 hour before shift change during 8 weeks 2) Control: waiting list control
Outcomes	Depression Anxiety Stress Scale (DASS‐21) and salivary alpha‐amylase; work satisfaction
Notes	Based on abstract only
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	"..were randomized to intervention or wait‐list control groups, stratified by gender.."
Allocation concealment (selection bias)	Unclear risk	Not reported
Blinding (performance bias and detection bias)   All outcomes	High risk	Not possible, self report
Incomplete outcome data (attrition bias)   All outcomes	Unclear risk	Not reported
Selective reporting (reporting bias)	Unclear risk	Not reported
Other bias	Unclear risk	Based on abstract only

Outcome or subgroup title	No. of studies	No. of participants	Statistical method	Effect size
1 Maslach Burnout Inventory (follow‐up less than 1 month)	1		Std. Mean Difference (IV, Fixed, 95% CI)	Totals not selected
1.1 Emotional exhaustion	1		Std. Mean Difference (IV, Fixed, 95% CI)	0.0 [0.0, 0.0]
1.2 Depersonalisation/ Disengagement	1		Std. Mean Difference (IV, Fixed, 95% CI)	0.0 [0.0, 0.0]
1.3 Personal Accomplishment (lack of)	1		Std. Mean Difference (IV, Fixed, 95% CI)	0.0 [0.0, 0.0]
2 Maslach Burnout Inventory (follow‐up 1 ‐ 6 months)	2		Std. Mean Difference (IV, Fixed, 95% CI)	Totals not selected
2.1 Emotional exhaustion	2		Std. Mean Difference (IV, Fixed, 95% CI)	0.0 [0.0, 0.0]
2.2 Depersonalisation	2		Std. Mean Difference (IV, Fixed, 95% CI)	0.0 [0.0, 0.0]
3 Maslach Burnout Inventory or Oldenburg Burnout Inventory (follow‐up more than 6 months)	3		Std. Mean Difference (IV, Fixed, 95% CI)	Subtotals only
3.1 Emotional exhaustion	3	313	Std. Mean Difference (IV, Fixed, 95% CI)	‐0.27 [‐0.50, ‐0.04]
3.2 Depersonalisation/ Disengagement	3	313	Std. Mean Difference (IV, Fixed, 95% CI)	‐0.33 [‐0.56, ‐0.10]
3.3 Personal Accomplishment (lack of)	2	182	Std. Mean Difference (IV, Fixed, 95% CI)	0.08 [‐0.22, 0.38]

Outcome or subgroup title	No. of studies	No. of participants	Statistical method	Effect size
1 Any stress outcome (follow‐up to 1 month)	4	97	Std. Mean Difference (IV, Fixed, 95% CI)	‐0.48 [‐0.89, ‐0.08]
1.1 Nurses	2	43	Std. Mean Difference (IV, Fixed, 95% CI)	‐0.70 [‐1.32, ‐0.07]
1.2 Physicians	1	38	Std. Mean Difference (IV, Fixed, 95% CI)	‐0.18 [‐0.81, 0.46]
1.3 Other Professionals	1	16	Std. Mean Difference (IV, Fixed, 95% CI)	‐0.70 [‐1.71, 0.32]
2 Any stress outcome (follow‐up 1 ‐ 6 months)	12	521	Std. Mean Difference (IV, Random, 95% CI)	‐0.49 [‐0.78, ‐0.21]
2.1 Nurses	7	288	Std. Mean Difference (IV, Random, 95% CI)	‐0.59 [‐1.02, ‐0.16]
2.2 Physicians	0	0	Std. Mean Difference (IV, Random, 95% CI)	0.0 [0.0, 0.0]
2.3 All Staff	5	233	Std. Mean Difference (IV, Random, 95% CI)	‐0.36 [‐0.71, ‐0.01]
2.4 Other Professionals	0	0	Std. Mean Difference (IV, Random, 95% CI)	0.0 [0.0, 0.0]

Methods	RCT, Canada
Participants	Physicians at 5 medical ICUs in academic medical hospitals in USA. Age 41 ± 6 years, Sex 76% men, Experience 8 ± ‐6 years. N = 39
Interventions	Experimental: Interrupted work schedule (IS) in which the weekends were taken over by someone else;  Control: In the continuous schedule (CS) one intensivist was responsible for 14 days; duration 9 months. 14 participants took IS only, 12 took CS only and 13 alternated both schedules; effectively comparing 27 IS schedules with 25 CS schedules
Outcomes	Burnout (on a scale from 5 ‐ 25) based on 5 questions similar to the MBI EE‐scale from the National Study of the Changing Workforce, a survey of United States workers performed by the Families and Work Institute, and formerly by the U.S. Department of Labor; also Job distress (on a scale from 6 ‐ 30) was measured but we did not use that scale.
Notes
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	"Sites were randomized between CS‐IS‐CS and IS‐CS‐IS patterns"
Allocation concealment (selection bias)	Unclear risk	Not reported
Blinding (performance bias and detection bias)   All outcomes	High risk	Not possible
Incomplete outcome data (attrition bias)   All outcomes	High risk	2 units dropped out after randomisation and the authors made no attempt to account for this
Selective reporting (reporting bias)	Low risk	There was no difference between Methods and Results.
Other bias	High risk	There were many more physicians in one of the intervention arms.

Methods	Controlled clinical trial, Canada
Participants	"The study population was composed of all care‐providing personnel in the experimental (n = 674) and control hospitals (n = 894), both of which offer general and specialised short‐term care. The population included all healthcare professionals in direct contact with patients (nurses and beneficiary attendants)." (p. 480). Of these, 467 participants remained at follow‐up (247 in the intervention group and 220 in the control group).
Interventions	"The intervention was defined as changes undertaken by the hospital to reduce adverse psychosocial factors in the workplace. Solutions proposed by the intervention team and adopted by the nursing department as well as any other objective change introduced with the explicit goal (or actual consequence) of improving one of the four targeted psychosocial factors were considered part of the intervention." (p. 480)
Outcomes	Client burnout, work burnout, personal burnout and psychological distress. In addition, the authors measured several intermediary outcomes such as psychological demands and decision latitude.
Notes
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	High risk	"This research used a before‐and‐after quasi‐experimental design with a control group." (p. 480)
Allocation concealment (selection bias)	High risk	There was no allocation concealment.
Blinding (performance bias and detection bias)   All outcomes	High risk	Not possible
Incomplete outcome data (attrition bias)   All outcomes	Unclear risk	Not reported
Selective reporting (reporting bias)	Low risk	All the measured outcomes were reported.
Other bias	Unclear risk	"It is difficult to identify which aspect of the intervention was responsible for the outcomes observed. In fact, even if the qualitative part of the study allowed to gather detailed information on what changes were made, still the design of the study was not experimental and as in most intervention research, changes in the workplace were beyond the researchers’ control." (pp. 483 ‐ 4)

Methods	RCT, UK
Participants	53 mental health nurses from 2 hospitals
Interventions	1) Experimental: Social support intervention: 5 consecutive weekly sessions of a minimum of 2 hours of group exercises underlining the importance of social support as key coping strategy when dealing with stress.  2) Control: Feedback only
Outcomes	The DeVilliers Carson Leary Stress Scale, The General Health Questionnaire (GHQ‐28), The MBI
Notes
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	"After allocation using sealed envelopes with random permuted blocks..." (p.33)
Allocation concealment (selection bias)	Low risk	"After allocation using sealed envelopes with random permuted blocks..." (p.33)
Blinding (performance bias and detection bias)   All outcomes	High risk	Not possible, self report
Incomplete outcome data (attrition bias)   All outcomes	High risk	"Attrition rates were similar in both conditions, at around 30%." (p. 31). Intention‐to‐treat analysis was not used.
Selective reporting (reporting bias)	Unclear risk	"...the Minnesota Job Satisfaction scale (Weiss et al., 1967) and the Cooper Coping Skills scale (Cooper et al., 1988b) were administered at Time 1 and Time 2." (p. 34) The results of these 2 outcomes were not reported.
Other bias	Unclear risk	We did not find any indications of other sources of bias.

Methods	RCT, Belgium
Participants	115 oncology nurses
Interventions	1) Experimental: Psychological Training Program: 3 weeks of training (each week including 5 consecutive days) and 1 week for the consecutive 3 months. Programme included 30‐hour theoretical information and 75 hours of role‐playing and experiential exchanges. The programme was designed to decrease nurses' professional stress levels, to improve nurses' attitudes and communication skills.  2) Control: No intervention
Outcomes	Nursing Stress Scale
Notes
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	Every time 20 nurses were enrolled, the nurses were randomly allocated to a training group (TG) or to a control group (CG).
Allocation concealment (selection bias)	Unclear risk	Not reported.
Blinding (performance bias and detection bias)   All outcomes	High risk	Not possible, self report
Incomplete outcome data (attrition bias)   All outcomes	Low risk	Only 1 participant dropped out of the intervention group and none from the control group.
Selective reporting (reporting bias)	Low risk	All outcomes reported.
Other bias	Unclear risk	We did not find any indications of other sources of bias.

Methods	RCT, Netherlands
Participants	99 nursing assistants
Interventions	1) Experimental: Integrated emotion‐oriented care: Basic training course of 2 days with an intermediary period of 2 weeks for homework (for all staff members on intervention wards) addressing staff members' own experience, phases of ego‐experience of the demented residents and the application of (non‐)verbal empathic skills Advanced course of 7 days spread out over 7 ‐ 8 months for 5 people from each intervention ward and an Adviser course of 10 days over 9 months for 1 person from each intervention ward.  2) Control: Training and support in giving usual care
Outcomes	The Organization and Stress Scale
Notes
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	"A pretest‐posttest control group design with matched groups (randomized clinical trial) was used" (p. 331)
Allocation concealment (selection bias)	Unclear risk	Not reported.
Blinding (performance bias and detection bias)   All outcomes	High risk	Not possible, self report
Incomplete outcome data (attrition bias)   All outcomes	High risk	"During the experimental period 25 nursing assistants dropped out due to: illness (11), pregnancy (2), and transfer (9). In three cases questionnaires were missing. Data analysis was carried out on 99 'complete' cases. Drop‐out did not differ between the groups..." (p. 333)
Selective reporting (reporting bias)	High risk	For nursing assistants results consist of covariance analyses that were not prespecified.
Other bias	Unclear risk	We did not any indications of other sources of bias.

Methods	RCT, Iran
Participants	45 male and female nurses with B. Sc. of nursing, who exclusively worked in emergency or acute, chronic, or specialised male and female wards of psychiatry, and who scored less than 150 on the Holms and Rahe stress scale. 66.7% were married, 73.3% were on rotating shifts, and 55.4% were in the age range of 25 ‐ 30.
Interventions	1) Experimental: Communication skills training for 6 hours over 3 weeks 2) Control: No intervention
Outcomes	A stress questionnaire designed by the researcher. The questionnaire was based on Tuft‐Anderson’s questionnaire, psychiatric nurses occupational stress scale (PNOSS), and an opinion poll of some nurses working in psychiatry wards about the sources of their stress. " To determine the validity of the questionnaire designed by the researcher, it was revised and approved by five faculty members of psychiatric nursing, one PhD of nursing, three psychiatric nurses, two psychiatrists, and six psychologists." (p. 397) "The questionnaire consisted of 34 questions, on a four point scale, ranging from 0 to 3, in which 0 implied no stress, 1 mild stress, 2 moderate stress, and 3 high stress. Thus, the highest and the lowest possible scores of the questionnaire were 102 and zero, respectively. The questionnaire covered six fields; nine items on coping with threatening cases, four items on shortage of resources, nine items on personnel conflicts, five items on planning issues, four items on working load, and three items on un‐preparedness for the occupational stress of psychiatric wards. Obtaining scores equal to or less than 30 was interpreted as low stress, while stress score of 40‐62, and 63 and above indicated moderate and high stress, respectively." (p. 397)
Notes
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	"the participants were randomly assigned to two groups; experiment and control. According to the random number table, of the 23 participants in the experiment group, four (17.4%) and 19 (82.6%) were from Noor and Farabi hospitals, respectively, while of the 22 participants in the control group, 10 (45.5%) and 12 (54.5%) were from Noor and Farabi hospitals, respectively." (p. 397)
Allocation concealment (selection bias)	Unclear risk	Not reported
Blinding (performance bias and detection bias)   All outcomes	High risk	Not possible, self report
Incomplete outcome data (attrition bias)   All outcomes	Unclear risk	The authors do not report about participants dropping out.
Selective reporting (reporting bias)	Unclear risk	The authors report the results of their stress questionnaire only as group means and t‐test P values.
Other bias	High risk	The only outcome measure the authors used was of their own devising and it was validated only by revision and approval by colleagues.

Methods	Cluster‐RCT, Netherlands
Participants	Nurses on wards of an academic hospital were screened for work and health problems: Experimental: 29 wards, 591 participants of which 151 screened positive Control: 28 wards, 561 participants of which 161 screened positive. Experimental: 17% Control 22% men, > 45 years age Experimental 51% Control 46%, > 10 years of experience Experimental 51% Control 41%
Interventions	1) Experimental: All who screened positive were referred to Occupational Health Physician (OHP). Participants who were screened as positive were invited for a face‐to‐face preventive consultation with their occupational physician. The consultation was voluntary, and workers could reschedule or cancel it if they wished. Supervisors were not informed about the screening results or about the invitation for and content of the preventive consultation of any employee. The 7‐step protocol for OHPs closely followed occupational physicians’ care as usual for consultations initiated by the employee in contrast to the compulsory consultation in the context of absenteeism. Occupational physicians received 3 hours of training from the researchers on the use of the protocol. (CBT) 2) Control: Waiting list: In the control arm. Participants filled out the baseline questionnaire; however, results of the screening‐questionnaires were not to be reported back to participants, and no further interventions were advised at baseline. As compensation, participants in the control arm received their personal screening results together with a tailored choice for a self‐help EMH intervention six months after baseline.
Outcomes	The study's primary outcome was help‐seeking behaviour; we used secondary outcomes: distress from the Dutch 4DKL, anxiety and depression from Brief Symptom Inventory
Notes	We got the following data for the distress scale of the 4DKL at 6 months follow‐up for the group who screened positive from author K Nieuwenhuijsen: Experimental: N = 86 6.24 ± 6.52 Control: N = 116 6.82 ± 6.57
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	" Randomization was performed at the ward level (n = 86). Randomization sequences with a block size of three were generated with Nquery Advisor (Statistical Solutions, Ltd, Cork, Ireland) by one researcher (K.N.) who was not involved in the recruitment"
Allocation concealment (selection bias)	Unclear risk	Not blinded
Blinding (performance bias and detection bias)   All outcomes	High risk	Providers not blinded, outcome self‐reported measures of stress
Incomplete outcome data (attrition bias)   All outcomes	High risk	At 3 months lost to follow‐up: Experimental 37% / Control 30%; at 6 months Experimental 46% / Contr 34%
Selective reporting (reporting bias)	Low risk	All outcomes reported that were announced in protocol
Other bias	High risk	Compliance very low: 34% of those invited visited their OHP

Methods	RCT, Turkey
Participants	" All of the nurses (n = 227) were invited to complete the Maslach Burnout Inventory (MBI) developed by Maslach & Jackson (1981). Those who completed the questionnaire and received a score on emotional exhaustion higher than the median score for all nurses were invited to participate in the burnout reduction intervention." (p. 487) 108 nurses were randomised to one of three conditions.
Interventions	1) Coping training (N = 36) "The group that received coping training consisted of two groups, each group consisting of 18 people. In the ﬁrst week, the concept of stress was explained to the nurses, and coping methods used by the nurses in stressful conditions were discussed. In the second session, basic communication skills on the stress level were discussed. In the third session, cognitive coping methods were presented theoretically. In the fourth session, cognitive distortions found among nurses and methods for coping with these distortions were discussed. In the ﬁfth session, the problem‐solving method was theoretically explained to the nurses. In the sixth session, stressful situations that the nurses encounter were discussed and resolved by means of the problem solving method. In the seventh session, problems that the nurses had difﬁculty coping with were discussed by utilizing the skills learned during the course of the programme." (p. 488) 2) Support group (N = 36) "...the support group consisted of three groups, each group consisting of 12 people. The nurses talked about the most frequently encountered stressors in the workplace and expressed their feelings towards their jobs. At the beginning of each session, the nurses expressed their feelings related to difﬁcult situations at the workplace. Then, a problem chosen by the nurses was attempted to be solved by using reﬂective cycle steps. Researchers provided information when needed. Possible solution methods were discussed in the groups, and the nurses were advised to use these methods in their daily lives. The nurses shared their difﬁcult and favourable times and also exchanged recommendations with each other." (p. 488) 3) Control: No intervention (N = 36)
Outcomes	MBI
Notes
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	"Random allocation was concealed by using a system of sequentially numbered, opaque, sealed envelopes containing the computer‐generated random allocation, which had been drawn up by a statistician. During the randomization, the researchers and the participants did not know the groups to which they would be allocated." (p. 487)
Allocation concealment (selection bias)	Low risk	See above
Blinding (performance bias and detection bias)   All outcomes	High risk	"No blinding was applied to the participants and the researchers." (p. 487)
Incomplete outcome data (attrition bias)   All outcomes	Low risk	"Intention‐to‐treat analysis was used because of sample loss." (p. 487)
Selective reporting (reporting bias)	Low risk	The authors only measured and adequately reported results of the MBI.
Other bias	Unclear risk	We did not find any indications of other sources of bias.

PERMALINK

Preventing occupational stress in healthcare workers

Jani H Ruotsalainen

Jos H Verbeek

Albert Mariné

Consol Serra

Abstract

Background

Objectives

Search methods

Selection criteria

Data collection and analysis

Main results

Authors' conclusions

Plain language summary

Summary of findings

Background

Description of the condition

Description of the intervention

How the intervention might work

Why it is important to do this review

Objectives

Methods

Criteria for considering studies for this review

Types of studies

Types of participants

Types of interventions

Types of outcome measures

Primary outcomes

Secondary outcomes

Search methods for identification of studies

Electronic searches

Searching other resources

Data collection and analysis

Selection of studies

Data extraction and management

Assessment of risk of bias in included studies

Measures of treatment effect

Unit of analysis issues

Dealing with missing data

Assessment of heterogeneity

Assessment of reporting biases

Data synthesis

Subgroup analysis and investigation of heterogeneity

Sensitivity analysis

Results

Description of studies

Results of the search

1.

Included studies

Excluded studies

Risk of bias in included studies

2.

Allocation

Blinding

Incomplete outcome data

Selective reporting

Other potential sources of bias

Effects of interventions

Summary of findings for the main comparison. Cognitive‐behavioural intervention compared to no Intervention for reducing stress.

Summary of findings 2. Relaxation compared to no intervention (SMD) for reducing stress.

Cognitive‐behavioural interventions

1. Cognitive‐behavioural intervention vs. no intervention

1.1. Any stress‐related outcome (Follow‐up to one month)

1.1.1 Cognitive‐behavioural intervention only vs. no intervention

1.1. Analysis.

1.1.2 Cognitive‐behavioural intervention and relaxation vs. no intervention

3.

1.2 Any stress‐related outcome (Follow‐up from one to six months)

1.2.1 Cognitive‐behavioural intervention only vs. no intervention

1.2. Analysis.

1.2.2 Cognitive‐behavioural intervention and relaxation vs. no intervention

4.

1.3 Any stress‐related outcome (Follow‐up more than six months)

1.3.1 Cognitive‐behavioural intervention with or without relaxation vs. no intervention

1.3. Analysis.

1.4 Anxiety (Follow‐up to one month)

1.4. Analysis.

1.5 Anxiety (Follow‐up one to six months)

1.5. Analysis.

Methods	RCT, Norway
Participants	32 nurses working at 2 psychiatric hospitals
Interventions	1) Experimental: Full‐body massage: 6 consecutive weekly 90‐minute full‐body aromatherapy massages.  2) Control: No intervention
Outcomes	Cooper's Job Stress Questionnaire
Notes
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	"Thirty‐five subjects registered for the study and were randomised by drawing numbers, to either a treatment group (n = 18) or a control group." (p. 91)
Allocation concealment (selection bias)	Unclear risk	Not reported.
Blinding (performance bias and detection bias)   All outcomes	High risk	Not reported, self‐report blinding not possible
Incomplete outcome data (attrition bias)   All outcomes	Unclear risk	"Three persons dropped out from the control group, two moved to other parts of Norway and one went on vacation" (p. 91)
Selective reporting (reporting bias)	Low risk	There was only one outcome measure and its results were reported
Other bias	Unclear risk	We did not find any indications of other sources of bias

Methods	cluster‐RCT, Denmark
Participants	210 eldercare workers
Interventions	1) Experimental 1: Stress Management Intervention: The SMI was developed to address the work stress in health care with particular attention to prevention of burnout and development of strategies for stress management. Training occurred over 20 weeks, with group sessions every 2 weeks, and each session lasting 2 hours. Between sessions, the participants were given assignments concerning implementation of the programme in daily practice.  2) Experimemtal 2:Transfer Technique Intervention: The TTI was based on the Stockholm training concept, which aims to reduce the biomechanical load on the back, minimise work in asymmetric postures, and prevent sudden unexpected loads. Training in the TTI arm was a combination of practical classroom education (24 hours for each worker) and instruction at the work site. There were 11 instructors who belonged to the 7 groups in the TTI arm, with 1 – 2 persons in each group who received 30 hours of education during the initial phase of the study.  3) Control: Reference Programme consisting of lessons of the participants' own choice in matters unrelated to the intervention programmes but of the same duration as the active intervention lessons (e.g. on skin care, proper treatment of a person with diabetes, etc.)
Outcomes	MBI (results not reported in article but obtained directly from author)
Notes
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	"Randomization was performed at group level because the intervention programs were meant to involve the employee as a group during education and implementation. The assignment to the different intervention programs was balanced to secure representation of all 3 programs in each of the wards." (p.1762)
Allocation concealment (selection bias)	Unclear risk	Not reported.
Blinding (performance bias and detection bias)   All outcomes	High risk	Not possible, self report
Incomplete outcome data (attrition bias)   All outcomes	High risk	"We observed no differences in age and number of years occupied in health care and mean intensity of LBP during the past year between participants who remained in the study and participants who dropped out." (p.1762)
Selective reporting (reporting bias)	High risk	Results data for the MBI, Setterlind's Stress Scores and rating of social support were not reported because they were not statistically significantly different between groups. "...[N]o significant changes were found in either of the intervention arms in ... the Maslach Burnout Inventory, the Setterlind stress scores, or the rating of social support (data not shown)" (p. 1765)
Other bias	Unclear risk	We did not find any indications of other sources of bias.

Methods	RCT, Brazil
Participants	75 nurses at the Teaching Hospital of the University of São Paulo. "In order to define the sample of participants, the authors used the Stress Inventory or Stress Symptoms List – SSL. This instrument was applied to all subjects who agreed to participate in the study (N=109); however, only subjects who achieved mean (29 to 60 points), high (61 to 120 points) or very high (>120 points) scores were included in the sample; 75 of them completed the study. As for the distribution of the participants, 22 subjects were placed in the Control Group, 27 in the Needles Group and 26 in the Seeds Group." (p. 88) "The inclusion criteria were: belonging to the nursing team; voluntary participation in the study with avaiability to attend the sessions; obtaining a minimum SSL score at mean, high and very high stress level; not being pregnant. The authors excluded from the sample all the subjects who went on vacation or medical leave after the  beginning of the study; did not show up to the session or gave up due to adverse effects, and those who had low SSL score." (p. 88)
Interventions	1) Experimental 1: Auriculotherapy (form of acupuncture performed on the ears) with needles (n = 27) 2) Experimental 2: Auriculotherapy with seeds (n = 26) 3) Control: No intervention(n = 22) "The intervention groups received eight sessions (one session a week), with duration of 5 to 10 minutes each session, on the Shenmen, Kidney and Brainstem points. The first two points have calmative properties and the kidney point has energetic function. After the location of the reactive points with a point locator, the ear auricle was hygienized with cotton and ethyl alcohol 70% and, then, semi‐permanent needles were applied or seeds were fixed with adhesive plaster, according to the intervention group. In the group of auriculo therapy with seeds, mustard seeds were used and the participants were instructed to stimulate them three times a day, for 15 times, with moderate pressure. The volunteers were instructed to remove the needles or seeds 24 hours before the session and, in case there was any discomfort, itching or signs of allergy, they should remove them before that." (p. 88)
Outcomes	Stress Symptoms List
Notes
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	"This randomized controlled clinical experiment was performed with three groups..." (p. 88)
Allocation concealment (selection bias)	Unclear risk	Not reported
Blinding (performance bias and detection bias)   All outcomes	High risk	Blinding was not possible
Incomplete outcome data (attrition bias)   All outcomes	High risk	"...there was a loss of 34 subjects during the study. Seven professionals went on vacation after the beginning of the study and two on medical leave; 12 missed the session because they had forgotten it, due to traffic problems or the difficulty to reschedule it and seven did not show up for the first session. One participant gave up due to adverse effects, in this case, nightmares, and five exclusions were due to low score (1), not belonging to the nursing team (3), and not filling out properly the questionnaires (1)." (p. 89) The authors do not report how the dropouts were distributed among the study groups.
Selective reporting (reporting bias)	High risk	The authors present data separately for participants who had had high SSL scores to begin with in table 2 but not at all for participants with a moderate SSL score.
Other bias	Unclear risk	We did not find any indications of other sources of bias.

Methods	RCT, Hong Kong
Participants	38 nurses. "All ED nurses who met the following inclusion criteria were invited to participate in the MEDI program: (a) currently employed ED nurse and (b) Cantonese‐English speaking" (p. 20) "Exclusionary criteria were as follows: (a) substantial drug or alcohol abuse problems, (b) current suicidal ideation, (c) the presence of psychotic symptoms, or (d) currently taking psychiatric medication." (p. 20)
Interventions	1) Experimental: MEDI intervention (n = 18) "The MEDI program is a combination of four crucial components: to pay attention to each moment with non‐judgemental manner (Mindfulness), to regulate intense emotions (Emotion regulation), to tolerate emotional distress when change is sluggish or implausible (Distress toleration) and to be more efficient in resolving interpersonal conflicts (Interpersonal effectiveness)." (p. 13) "...the MEDI program, which included four sessions, 1.5h each concerning mindfulness, emotion regulation, distress tolerance, and interpersonal effectiveness." (p. 24) 2) Control: No intervention (n = 20)
Outcomes	MPSS‐R consisting of 4 subscales; OS = Occupational Stress, JD = Job Dissatisfaction, NPA = Negative patient attitude and SS = Somatic Stress.
Notes
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	"...two groups were formed: (a) MEDI treatment (n=18) and (b) no treatment control (n=20) based on each participant blindly drawing a number from an envelope (1 = MEDI group and 2 = control condition)." (p. 32)
Allocation concealment (selection bias)	Unclear risk	Authors do not report if allocation was concealed
Blinding (performance bias and detection bias)   All outcomes	High risk	Blinding of participants (also outcome assessors of self‐report measures) and intervention providers was not possible.
Incomplete outcome data (attrition bias)   All outcomes	High risk	Participants were "[e]xcluded from analysis due to drop out". (p. 32)
Selective reporting (reporting bias)	Low risk	All measured outcomes were reported.
Other bias	High risk	The dropout rate was 50% of the original sample (12/18 in intervention group and 7/20 in control group).

Methods	Cross‐over study, Taiwan
Participants	54 newly‐employed nurses who had been in their job for less than one year. "The 54 participants were all female with a mean age of 23.4 (SD = 2.46). Only one participant was married. The majority were registered nurses (n = 49, 90.7%), with no past clinical experience (n = 42, 77.8%) and most had graduated from junior college (n = 40, 74.1%)." (p. 2419) "To qualify for participation in the study, participants were required to: (1) have normal hearing without the use of any hearing aid; 2) have been employed full‐time as a nurse for <1 year; (3) be between 20 and 40 years of age; (4) have a self‐rated VAS 0‐10 stress score of ≥6 and (5) weigh between 45 and 70 kg." (p. 2416) "Exclusion criteria included: (1) current or past abnormalities in endocrinology or metabolism (Lavie et al. 2009); (2) pregnancy; (3) chronic organic disease; (4) medicine, alcohol or tobacco use; (5) those who have experienced a major stressful event in the past 6 months (The Social Readjustment Rating Scale) (Holmes & Rahe 1967); and (6) those receiving physical training (Schulz et al. 2004)." (p. 2416)
Interventions	"Interventions consisted of an alternating music and chair rest or chair rest and music sequence" (p. 2416) The procedure took one hour and 20 minutes to complete. It started with 30 minutes listening to music whilst seated, followed by a 20‐minute washout during which participants "...were asked to get up and move around while the researcher made conversation with the participant based on a standardized chatting guideline about the daily leisure activities and interests participants had outlined beforehand" and finally participants rested whilst seated for another 30 minutes. Half the participants were randomised to receive the interventions the other way round.
Outcomes	Self‐perceived stress on a 10 cm VAS, Heart rate, mean arterial pressure and ﬁnger temperature.
Notes	Authors could not be traced for inquiries
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	"A randomized controlled crossover clinical trial was used to examine the effectiveness of music on stress reduction." (p. 2416)
Allocation concealment (selection bias)	Unclear risk	Not reported.
Blinding (performance bias and detection bias)   All outcomes	High risk	Not reported but not possible.
Incomplete outcome data (attrition bias)   All outcomes	Low risk	Apparently there were no dropouts so all data are accounted for.
Selective reporting (reporting bias)	Low risk	All the measured outcomes were reported.
Other bias	High risk	The cross‐over design may be a source of bias. It is impossible to say if a 20‐minute washout is enough.

Methods	RCT, Taiwan
Participants	60 hospital nurses suffering from either: insomnia, headache or gastrointestinal discomfort
Interventions	1) Experimental: Assertiveness training: 6 2‐hour sessions on Monday, Wednesday and Friday at 2pm ‐ 4pm on t2consecutive weeks. The contents of sessions included the concept of beliefs and negative self‐statements, building a positive belief system, applying assertion to clinical settings and developing group and self‐reinforcement support systems.  2) Control: Traditional in‐service programme about computer applications in nursing
Outcomes	Perceived Stress Scale, Rathus Assertiveness Schedule
Notes
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	"Subjects were randomly assigned to one of two treatments: assertiveness training (AT) or alternate treatment control (ATC), which served as a control and contained updated knowledge of new computer technology for in patient settings." (p. 419)
Allocation concealment (selection bias)	Low risk	"Subjects admitted to the study agreed to random treatment assignment and a 2‐month commitment to the study. However, the subjects did not know whether they would receive treatment or control procedures during that time." (p. 425)
Blinding (performance bias and detection bias)   All outcomes	High risk	Not possible, self report
Incomplete outcome data (attrition bias)   All outcomes	High risk	"Of the respondents who initially chose to participate, three did not complete the study and were not included in the data analysis due to their failure to attend all sessions, failure to complete the questionnaire, or decision to leave hospital employment." (p. 425)
Selective reporting (reporting bias)	Low risk	All outcomes reported
Other bias	Unclear risk	We did not find any indications of other sources of bias.

Methods	Controlled Clinical Trial, Taiwan
Participants	49 junior nursing students engaged in a medical‐surgical rotation from a university. "The experimental groups consisted of mentor and mentee students; each mentor was paired with one mentee. The mentors were recruited students who had received medical–surgical experience before entering university. They also have a junior college nursing certiﬁcation and a Registered Nursing license (n = 17). The mentees (n = 17) have had no prior medical–surgical clinical practice experience. The control groups consisted only of students who had no medical or surgical clinical practice experience (n = 32). The control group and experimental group were assigned in two different clinical settings at different time in order to minimize within‐group inﬂuence and its interference with the research." (p. 205)
Interventions	1) Experimental: Peer mentoring intervention, 3 days per week for 4 weeks (n = 17). "The mentors were selected from the mentee’s class. The mentors were all informed in advance about the aim and process of this study. They were volunteers and were requested to be motivated, self‐confident, enthusiastic and open, especially when dealing with mentees. They were supposed to be the role models for their mentee when taking care of patients or interacting with the clinical teacher and medical staff. The mentor also played a psychosocial role, for example, listening to the mentees’ complaints,sharing their pre clinical practice experience and generally giving all‐round support." (p. 205) "Mentors were coached and supervised by the clinical teacher to make sure of the accuracy and reliability of their knowledge when they first practiced their nursing skills on an actual patient. After that, mentors could assist mentees in some basic and non‐intervened nursing skills, such as naso‐gastric tube care and mouth care, which had been taught in school lectures. Mentors and mentees could also learn to understand each other’s patients through communicating with the patients and discussing with the medical staff. They were able to converse with each other at least once a week during the rotation. In addition, after work, they gave support through correspondence, over the telephone or e‐mail." (p. 205) 2) Control: (n = 32) No intervention
Outcomes	The Perceived Stress Scale (PSS) Total score
Notes
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	High risk	"The control group and experimental group were assigned in two different clinical settings at different time in order to minimize within‐group influence and its interference with the research." (p. 205)
Allocation concealment (selection bias)	High risk	Allocation was not concealed.
Blinding (performance bias and detection bias)   All outcomes	High risk	Not possible, self report
Incomplete outcome data (attrition bias)   All outcomes	Low risk	Apparently none of the participants dropped out.
Selective reporting (reporting bias)	Unclear risk	The study recruited 34 students in the experimental group (17 mentors and 17 mentees) but the authors report outcome data only for 17 of these, presumably the mentees.
Other bias	High risk	The authors state that "… gender will be a confounding variable in this study" (206)

Methods	Cluster‐randomised cross‐over non‐inferiority trial, USA
Participants	62 hospital physicians
Interventions	1) Experimental: 2‐week hypothesised to be least stressful  2) Control: 4‐week rotations to improve education and supervisor relation
Outcomes	Burnout was measured with a questionnaire that contained: "...8 items from Minimizing Error, Maximizing Outcomes...and 4 items from the short version of the Perceived Stress Scale. Emotional exhaustion... was measured using 9 items from the Maslach Burnout Inventory Human Services Survey and a single‐item measure from the National Job Burnout Survey that uses a personal definition of burnout." (p. 2201) We used the MBI emotional exhaustion scale. The study´s primary outcome was patient re‐hospitalisation.
Notes	The authors kindly provided the continuous outcomes for the MBI scale. We used the data at physician level not at work schedule level.
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	"After dividing each attending physician’s year‐long service commitment into 4‐week segments, we used a computerized random‐number generator to block randomize these 4‐week segments (as single 4‐week or paired 2‐week rotations) using block sizes of 2, the number of rotation types." (eAppendix)
Allocation concealment (selection bias)	High risk	Not concealed
Blinding (performance bias and detection bias)   All outcomes	High risk	"Attending physicians were not blinded to their rotation assignments because the year long rotation schedule was distributed 3 months before the study commenced. Attending physicians, trainees, and patients were, nonetheless, blinded to the study hypothesis." (p. 2200‐2201)
Incomplete outcome data (attrition bias)   All outcomes	Low risk	There were very little missing data. "A severity assessment from a 2‐week rotation was missing from one attending physician who had completed 5 other assessments after 2‐week rotations."
Selective reporting (reporting bias)	Low risk	All outcomes listed in the Methods section were reported.
Other bias	Unclear risk	We did not find any indications of other sources of bias.

Methods	Cluster‐RCT, Sweden
Participants	26 health care personnel in a geriatric hospital
Interventions	1) Experimental: Support and advice from a psychologist: 1 hour every second week for 20 weeks regarding knowledge about stressors, relaxation techniques, life style factors and physiological, cognitive, emotional and behavioural stress reactions. Participants were encouraged to alter reaction patterns and to discuss and test new alternative patterns and enhance own work conditions and increase self control over work processes.  2) Control: Passive attendance of psychologist in staff meetings
Outcomes	A structured made‐to‐measure stress questionnaire
Notes
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	"The wards were randomly allocated to be either intervention (I) or control (C) ward" (p. 80)
Allocation concealment (selection bias)	Unclear risk	Not reported.
Blinding (performance bias and detection bias)   All outcomes	High risk	Not possible, self report
Incomplete outcome data (attrition bias)   All outcomes	Low risk	"There were no dropouts during the study period" (p. 81)
Selective reporting (reporting bias)	Low risk	All outcomes reported.
Other bias	Unclear risk	We did not find any indications of other sources of bias.

Methods	RCT, Argentina
Participants	74 hospital paediatric resident physicians. " A total of 81% were female; the mean age was 27.3±1.4 years; 57% were working in inpatient areas, 35% in the outpatient clinic, and 8% in the intensive care unit." " A comparison of the characteristics of both groups (experimental and control) revealed no significant  differences." (p. 494)
Interventions	1) Experimental: Self‐care workshop intervention (n = 37). "The experimental group received a brief intervention consisting of two 2.5‐hour workshops directed by mental health professionals, which covered repercussions of  burnout syndrome on professional activity, recognition of risk indicators for burnout syndrome, and tools to cope (identification of strengths, coping behaviors, preventive and self‐care behaviors)." (p. 494) 2) Control: (n = 37) No intervention
Outcomes	MBI
Notes	It is unclear why 43 (37%) out of the available 117 resident physicians did not participate in the study.
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	"After administration of the questionnaire, subjects were randomly assigned to one of the two study groups." (p. 494)
Allocation concealment (selection bias)	Unclear risk	Not reported.
Blinding (performance bias and detection bias)   All outcomes	High risk	Participants could not be blinded regarding group allocation. Also as outcome measurement was by self report, there was no blinding
Incomplete outcome data (attrition bias)   All outcomes	Low risk	Apparently no participants were lost to follow‐up
Selective reporting (reporting bias)	High risk	The authors do not report standard deviations with the mean MBI subscale scores.
Other bias	Unclear risk	It is unclear if these 74 were all the participants or only those that could be followed up.