Realist synthesis of factors affecting retention of staff in UK adult mental health services

Abstract

Objectives

The shortage of healthcare staff is a global problem. UK mental health services have, on average, a higher turnover of staff than the NHS. Factors affecting retention of this staff group need to be explored in more depth to understand what is working for whom, for what reasons and in what circumstances. This review aims to conduct a realist synthesis to explore evidence from published studies, together with stakeholder involvement to develop programme theories that hypothesise how and why retention occurs in the mental health workforce and identify additional evidence to explore and test these theories thereby highlighting any persistent gaps in understanding. This paper develops programme theories that hypothesise why retention occurs and in what context and tests these theories thereby highlighting any persistent gaps in understanding.

Methods

Realist synthesis was used to develop programme theories for factors affecting retention of UK mental health staff. This involved: (1) stakeholder consultation and literature scoping to develop initial programme theories; (2) structured searches across six databases to identify 85 included relevant literature relating to the programme theories; and (3) analysis and synthesis to build and refine a final programme theory and logic model.

Results

Phase I combined findings from 32 stakeholders and 24 publications to develop six initial programme theories. Phases II and III identified and synthesised evidence from 88 publications into three overarching programme theories stemming from organisational culture: interconnectedness of workload and quality of care, investment in staff support and development and involvement of staff and service users in policies and practice.

Conclusions

Organisational culture was found to have a key underpinning effect on retention of mental health staff. This can be modified but staff need to be well supported and feel involved to derive satisfaction from their roles. Manageable workloads and being able to deliver good quality care were also key.

Strengths and limitations of this study.

• Realist synthesis focusses on interpretation rather than comprehensiveness.

• The use of stakeholders early in the process was a strength.

• The use of concepts rather than specific interventions in the synthesis improves the generalisability of the findings.

Introduction

In 2022, the WHO called for all nations to invest in mental health workforce, with many facing shortages of trained mental health staff, with poor retention rates and low up take of new staff being trained.¹ Shortages are being seen across mental health professions, for example across much of the world’s population there is just one psychiatrist to serve around 200 000 people, with the most severe shortages seen in rural areas where other mental healthcare providers who are trained to use psychosocial interventions are even scarcer.²

The problem of workforce retention in the UK National Health Service (NHS) is widely acknowledged and has worsened every year since 2011.³ UK mental health services have, on average, a higher turnover of staff than the NHS as a whole (13.4% of all mental health staff left in 2018/2019 compared with 11.9% across all NHS employers) and more vacant positions.^{4 5} Job satisfaction for mental health nurses is reportedly the lowest it has ever been, with many nurses emotionally and physically exhausted, leading them to consider leaving their profession.⁶ In December 2021, there were 1 110 000 vacancies in the NHS, with 22% of all vacancies in mental health trusts despite only 14% of NHS spending being on mental health.⁷ A recent report by the Royal Collage of Psychiatrist states that in England 9.3% vacancy rate in consultant posts and an increasing in use of locum consultant psychiatrist posts.⁸ These staff shortages are known to impact patient quality of care with approximately 1.5 million people in England waiting for mental health treatment and that two-fifths of those on mental health waiting lists, ended up receiving emergency or crisis services before receiving treatment.⁹ More immediate emphasis should be placed on retaining the staff already in post as retaining experienced staff is seen to not only have the potential to improve patient care and waiting times but also has the potential to inspire a new mental health workforce to join aiding retention and boost recruitment.⁶

Previous reviews have focused on the determinants of the healthcare workforce turnover and intention to leave rather than retention, within the physical healthcare workforce^10–12 or only focused on specific mental health professions such as mental health nurses¹³ and psychiatrists¹⁴ rather than the entire workforce. Turnover and intention to leave has been associated with organisational factors such as resource allocation, leadership and pay as well as individual factors including stress and job dissatisfaction.^{10 11} Stress experienced by healthcare staff appears linked to the ability to give time to patients in need, missed care and the perception of the quality of care delivered, all of which potentially impact job satisfaction an intent to leave.¹⁵ Job dissatisfaction appears to be also strongly associated with poor retention of mental health nurses¹³ and one of the key predictors of turnover of all healthcare staff. Leadership appears to have an indirect role, in that it affects the organisational culture (the set of beliefs, values and behaviours that determine an organisation’s identity) which in turn affects job satisfaction.¹⁶ For example a transformational leadership style is related to supportive work environments and staff intention to remain,¹⁷ while a supportive and communicative leadership style leading to organisational commitment is a strong predictor of low turnover.¹²

Many of the stressors highlighted by previous reviews will be also relevant to the mental health workforce such as limited resources, job dissatisfaction, pay or poor leadership, but some stressors effecting mental health workforce retention maybe specific to this setting¹³ such as the use of restraint and seclusion,¹⁸ dealing with verbally aggressive and physical violent patient acts,¹⁹ continuous monitoring of patients at risk of self-harm and suicide.²⁰ In 2018, the mental health workforce reported higher emotional exhaustion than emergency nurses and equal burnout to cancer professionals.²¹ Factors affecting all mental health staff retention need to be explored in more depth to understand what is working for whom, for what reasons and in what circumstances.

This review aims to generate an in-depth understanding of not only why mental health services find it challenging to retain its clinical staff, but what retention strategies are working and in what circumstance. The review’s objectives are to conduct a realist synthesis to explore evidence from published studies, together with stakeholder involvement to develop programme theories (PTs) that hypothesise how and why retention occurs in the mental health workforce and identify additional evidence to explore and test these theories thereby highlighting any persistent gaps in understanding.

Methods

We undertook a realist synthesis of the literature to create an overarching programme theory of the factors at an organisational, management and frontline level that influence retention and turnover of mental health staff in the UK.

Realist methodology

A realist review is a theory-led method which seeks to understand a particular outcome through exploring the relationships between the context and the underlying causal processes (mechanisms) that lead to that outcome.²² The realist review process is used to generate ‘If–Then–Leading to’ statements that provide possible explanations for the outcome of interest. For example, ‘IF management ensure that staff have the time and support required to deliver needed care (Context) THEN patients engage with their care (Mechanism) and staff feel their role is meaningful and important (Mechanism) LEADING TO increased job satisfaction (Outcome) and enhanced patient outcomes (Outcome(s)). These statements, also known as Context–Mechanism–Outcome Configurations (CMOCs), are subsequently refined to produce PTs, which are then synthesised into a single overarching theory to be tested against existing studies (synthesis) or primary research (evaluation).

The realist approach was appropriate for this review as it recognises that the context within which mental health workforce retention occurs is ‘complex, multifaceted and dynamic’.²³ Factors affecting this phenomenon/outcome are complex and changeable, involving various people, structures, policies and practice at different organisational levels. The effectiveness of any intervention to improve retention is contingent on diverse factors, and multiple explanations may be advanced for how these operate. This review seeks to identify the mechanisms that are activated across a variety of contexts operating at different levels (individual, team and organisational) within the healthcare system, which lead to outcomes relating to retention. Realist review methods enable exploration of this complexity and interconnectivity and enable development of a theoretical understanding of what factors influence retention within the UK mental health workforce.

The review was conducted in three phases¹: exploratory scoping of key literature and stakeholder consultation to identify initial PTs²; structured searches, screening and data extraction of included studies related to each identified PT; and³ analysis and synthesis leading to refinement of the PTs and development of a final overall PT and logic model.

No changes to the protocol were introduced during the review process. While the phases were sequential, each phase itself was iterative and went through rounds of refinement which ultimately resulted in new interpretations and the development of theory. The RAMESES (Realist and Meta-narrative Evidence Syntheses: Evolving Standards) publication standards informed the reporting of this review.²²

Patient and public involvement

Phase I of the study (described below) involved extensive patient and staff involvement in the design of the research questions, particularly around priorities and experiences and in the design of the study, particularly around the structured searches.

Phase I: exploratory scoping and stakeholder engagement

Exploratory scoping was conducted to locate and explore available evidence on what key factors might influence mental health workforce retention. Lines of inquiry included: exploring assumptions that underpin workforce retention interventions; the influence of job satisfaction on retention; and reported enablers or barriers to successful implementation of retention interventions at an organisation, management and frontline level. The purpose of this stage was to inform and shape subsequent search strategies and involved purposeful searching²⁴ of specific bodies of evidence based on the teams’ clinical knowledge and review expertise.

Search terms related to ‘healthcare personnel’ were variously combined with such terms as ‘retention, turnover, leavers, resignations and job satisfaction’ in the Cochrane and PubMed databases and Google Scholar for grey literature. The wider review team identified and prioritised information sources that could contribute to theory-building about factors influencing workforce retention. Twenty-seven sources (see table 1) were included, including policy and guidance documents, key research literature and grey literature. All reviewers initially independently extracted data from five sample papers to ensure consistency in approach and develop extraction tables. Findings were compared and discussed until consensus was reached. Data extraction of the remaining key papers was then divided between the team. Data were extracted as ‘If–Then–Leading to’ statements, that is, CMOCs. Extraction only occurred when at least two components of the If–Then–Leading to’ statements were present in a paper. Where complex ‘If–Then–Leading to’ statements were found with multiple components, these were extracted as individual statements on to an excel spreadsheet. When incomplete ‘If–Then–Leading to’ were extracted the team hypothesises on what the missing element might be during research meetings.

Table 1.

Scoping papers included in phase I

Area of retention (title screening)	Author and year	Study design	Workforce area	Focus
General retention strategies and policies	Brook et al 2019 *35	Systematic review	General healthcare workforce	Retention and turnover of early career nurses
	Loan-Clarke et al 201036	Longitudinal study (UK)	General healthcare workforce	Retention and turnover allied health professionals
	Dieleman et al 2011*37	Realist review	General healthcare workforce	Rural health workers retention
	Drennan et al 201538	Project report: mixed-methods (UK)	General healthcare workforce	Adult nurse turnover and retention
	Efendi et al 2019*39	Concept analysis review	General healthcare workforce	Nurse retention
	Halter et al 2017*40	Systematic review of systematic reviews	General healthcare workforce	Nursing turnover
Workforce well-being	Holland et al 201841	Cross sectional survey (Australia)	General healthcare workforce	Nurses and midwives well-being
	Johnson et al 2018*42	Narrative review	Mental healthcare workforce	Mental healthcare staff well‐being and burnout
	Morse et al 201243	Literature review	Mental healthcare workforce	Mental healthcare staff burnout
	Tomietto et al 201944	Cross-sectional validated questionnaire (European)	General healthcare workforce	Nurse well-being and engagement
Leadership	Hartviksen et al 201945	Systematic review and meta-synthesis	General healthcare workforce	Healthcare middle managers’ capability and capability
	Kleinman 200446	Literature review	General healthcare workforce	Leadership in staff nurse retention strategy
Workplace environment	Jelfs et al 201447	Literature review	General healthcare workforce	Retention strategies in healthcare organisations
	Redknap et al 201548	Literature review	Mental healthcare workforce	Mental health nurse retention and practice environment
	Twigg and McCullough 201449	Literature review	General healthcare workforce	Nurse retention and strategies to enhance positive practice environments
Workforce training and development	Edwards et al 201550	Systematic review	General healthcare workforce	Transition and support for student and preceptorship nurses
	Morse et al 201243	Literature review	Mental healthcare workforce	Burnout
	Williams et al 201651	Realist review	General healthcare workforce	Workforce development interventions
	Zhang et al 201652	Systematic review	General healthcare workforce	mentoring programme for newly graduated nurses
Workforce satisfaction and retention	Duffield et al 200953	Cross-sectional survey (Australia)	General healthcare workforce	Staff satisfaction and retention and nursing unit manager role
	Duffield et al 201154	Cross-sectional survey (Australia)	General healthcare workforce	Nursing unit manager, retention, and work environment.
	Rafferty et al 201855	Literature review	General healthcare workforce	Nurse drift and dilution
	Salie and Schlechter 201256	Evaluation (South Africa)	General workforce	Staff reward and recognition programme
Organisational factors	Okello and Gilson 201557	Systematic review	General healthcare workforce	Relationships and motivation in the health sector
Recruitment	Dolea et al 201058	Literature review	General healthcare workforce	Strategies to attract and retain of health workers in remote and rural areas
	Kroezen et al 201559	Review and case study (European)	General healthcare workforce	Recruitment and retention interventions
Safe staffing	Ball et al 201960	Survey, national data, case studies and realist evaluation (UK)	General healthcare workforce	Safe staffing, impact and implications

*Key papers.

Key contexts, mechanisms and outcomes across the data were identified and agreed by the research team²⁵ and mapped to explore their dynamic and interconnected nature.²⁶ From this mapping process, initial PTs (IPT) were drawn out and refined. These were then discussed and prioritised by the reviewers, the research management group and a small panel of clinical academics (n=4), to identify a shortlist of eight initial PTs.

Two stakeholder consultation workshops were then held to obtain feedback on and further prioritisation and development of the initial PTs. Stakeholders were defined as people with experience of working in or using mental health settings and were approached via local NHS organisations research groups, professional bodies and service user groups. Thirty-one stakeholders were involved in this process: 23 mental health professionals (including doctors, nurses, occupational therapists and clinical psychologists) and 8 members of a local service user group. Both groups also suggested refinements to the wording of PTs. Following this process, six refined PTs were identified for further exploration in phase II.

Phase II: structured searches, screening and data extraction

Searches

Informed by phase I, search strategies to focus on each of the prioritised PTs were developed by an information specialist, who undertook the subsequent searches between March and July 2020. Search terms relating to the mental health workforce, the UK and qualitative research were combined with specific terms relating to each PT for each of the six searches. Full details of the search strategies are in online supplemental appendix tables 1 and 2. Medicine, nursing and health databases were supplemented by social science sources to cover human resource and workload aspects of the review question. The following databases were searched for literature published from 2004 onwards: MEDLINE (including Medline-in-Process and Epub ahead of print); EMBASE; CINAHL; Cochrane Library; PsycINFO; and ProQuest social science databases (including ASSIA). We included only qualitative studies of mental health staff to ensure we focused on participants’ experiences and perceptions and explored rich contextual data. Duplicate references were removed. The search and screening for each PT was documented separately to enable tracking of individual records through the stages of the process.

Sifting/screening

The review team worked with the wider project team to develop overall inclusion and exclusion criteria, as shown in table 2. These were supplemented by additional PT-specific criteria.

Table 2.

Inclusion/exclusion criteria

	Included	Excluded
Type of literature	Peer-reviewed studies English language
	Primary qualitative research Qualitative components of mixed-methods studies	Systematic reviews Quantitative only studies
Setting/patients	Any adult mental health service including inpatient, outpatient and community Dementia services, if part of a mental health service Wider studies where mental health-specific data can be extracted	Children and Adolescent mental health services (CAMHS) Drug treatment and learning disability services, if not provided by mental health professionals Social care settings General healthcare settings providing treatment to mental health patients for example, Emergency department (ED), general pracitce (GP)
Staff groups	All grades and types of registered mental health professionals (including nurses, doctors, occupational therapists, psychiatrists)	Non-registered and non-clinical staff (eg, care assistants, porters, administrative staff)
Geographical area	UK, ie, England, Northern Ireland, Scotland, Wales or where UK-specific data can be extracted	All other countries
Date	2004, with particular focus on 2010 onwards

Four members of the team undertook title, abstract and then full text screening. Working in pairs they reviewed a common subset of approximately 20% of identified studies, comparing inclusion/exclusion and overall agreement. The high level of agreement from this process (95%) was considered justification for assigning the remaining screening to individual members of the team. Uncertainties regarding inclusion were brought to team meetings for discussion and resolution. The team met regularly throughout the screening process. Three key elements of richness, relevance and rigour were considered.²⁷ Screening was undertaken separately for each PT and items considered relevant to other PTs were added to the other PTs for full review. References of all full text papers were checked for additional eligible studies, which were then retrieved and screened.

Data extraction

Study characteristics were extracted into bespoke data extraction forms (one for each PT) by three members of the team. Data extracted included: authors, year, nation (England, Wales etc), patient group, setting, type and number of staff involved in the research, data collection method, findings, limitations, and illustrative quotes.

Where papers were relevant to multiple PTs data, relevant data were added simultaneously to the different PT data extraction tables. The quantity and richness of relevant data varied considerably, with only very limited data being relevant in some cases for example, mixed-methods studies or surveys with open questions. Judgements on study quality were deferred until later stages of analysis.

Team members then used the data extraction table to identify key factors and issues from each PT. These key themes, informed by team member knowledge, were then used to inform the next stage of the review.

Phase III: analysis and synthesis

Data synthesis was undertaken by three team members. Each team member independently summarised the key CMOCs from the data extraction and developed logic models to describe the PTs for which they had undertaken data extraction and were therefore most familiar with. This was done by starting with one CMOC and then looking for interconnections with others, until all had been incorporated into one large logic model. This was repeatedly refined to ensure overlapping elements were removed (eg, risk and safety are two sides of the same issue so were merged). Some of the CMOCs led on to each other meaning one outcome became the context for other CMOC. The logic model was refined to address this complexity. Findings were discussed within the wider team and logic model development was undertaken in consultation with the team methodologist.^{28 29} These discussions assisted the refinement of the PTs, ensuring they were plausible, grounded in the evidence and comprehensible. The team also reviewed whether the findings from this process suggested the need to modify or further develop the existing PTs, including identifying connections and overlaps between the PTs. As a result, the six individual PTs were refined and assimilated within three integrated key PTs which the team considered more accurately provided a whole systems perspective on the issues relating to staff retention. While other PTs could be seen to be operating, these three appeared to be the key drivers, and a single overarching logic model was then developed to illustrate the interconnectedness and complexity of these refined PTs. One team member initially drafted this model, which was progressively refined and simplified through multiple iterations within the review team and wider advisory team to clarify and accentuate the key findings. This process in turn led to the reformulating of the six ‘If–Then–Leading to’ statements created in phase I to create three new more clearly articulated core PTs. Throughout this process, the team ensured that the evolving PTs remained underpinned by the literature and informed by the findings of the previous stages of the review.

Results

Phase I findings: identification of initial PTs

Phase I led to prioritising of six PTs for further exploration in phase II searches: perceived quality of patient care delivered; workload and staffing levels; team relationships and cohesion; leadership; development opportunities; and supervision. These PTs were formally articulated in the CMOC format of IF–THEN–LEADING TO, see table 3.

Table 3.

Initial programme theories developed during exploratory scoping and stakeholder engagement

Initial programme theories (IF–THEN–LEADING TO statements)	Evidence from preliminary scoping and supporting references and stakeholder workshop
PT1 Perceived quality of patient care IF staff have the time and support to deliver the care to their patients that meets their expectations. THEN patients are more likely to engage, and staff will feel the job is meaningful and important LEADING TO improved job satisfaction and better patient outcomes.	Summary: Perceived quality of patient care increases job satisfaction. References: Brook et al, Dieleman, Johnson et al 2017; Williams et al 2016 (online supplemental appendix table 3) Stakeholders example quote: When the demands outstrip the capacity of the person to meet them burnout and failures in care are inevitable. This is a systemic problem though, not a failure of individual resilience; no one could possibly have the resilience to continuously meet these demands and expectations. Qualtrics 2020 (anonymous IAPT staff)
PT2 Leadership IF team leaders are trained to be good communicators and effective problem solvers THEN frontline staff will have good role models who are accessible and that they trust and can aspire to, LEADING TO increased confidence and job satisfaction.	Summary: Good Leadership leads to increased staff commitment, feeling valued and confidence in leadership. References: Ball et al 2019, Drennan et al 2015, Halter et al 2017, Hartviksen et al, Jelfs et al 2014, Johnson et al 2017, Kleinman 2004, Redknap et al 2015. Stakeholders example quote: feeling valued and getting positive feedback for good work - Padlet 2019 Anonymous research champion event. Recognition from management that we are doing a good job- Qualtrics 2020 Anonymous MH nurse
PT3 Supervision IF team leaders are able to offer regular, meaningful and supportive supervision THEN frontline staff will feel supported and less isolated LEADING to better job satisfaction and less stress	Summary: Supervision needs to be regular, meaningful and supportive to enable staff to feel supported and increase competence. References: Dolea et al 2010, Drennan et al 2015, Halter et al 2017, Jelfs et al 2014, Morse et al 2012 Stakeholders example quote: I have ranked supervision low but it probably should be higher. Mine is a waste of time, if it even happens. My supervisor (is) inconsistent and totally demotivating… I get my support from elsewhere - Qualtrics 2020 ANONYMOUS theory development stakeholder event.
PT4 Training and development IF an organisation offers a range of training and development opportunities for staff, WITH time/support allocated to complete, THEN staff will feel more competent and confident in their own abilities LEADING TO better perceived patient care and increase in job satisfaction.	Summary: Development opportunities leads to loyal, skilled, confident staff who want to work for the organisation that invests in them References: Ball et al 2019, Brook et al 2019, Dieleman et al 2011, Drennan et al 2015, Efendi et al 2018, Jelfs et al 2014, Kleinman et al 2004, Morse et al 2012, Williams et al 2016 Stakeholders example quote: I think feeling you are on a career trajectory is important and might need to be pulled out more explicitly. - Qualtrics 2020 Anonymous Allied health professional (AHP)
PT5 Workload IF an organisation (has the policies in place) to ensure all its teams are meeting safe staffing levels THEN staff will have the capacity to deliver the quality of care they expect to LEADING TO less burnout/emotional stress/increased retention.	Summary: Safe workload and staffing levels enable staff to deliver high quality care. References: Ball et al 2019, Duffield et al, Halter et al, Jelfs et al 2014, Johnson et al, Morse et al 2012 Stakeholders example quote: ‘When the demands outstrip the capacity of the person to meet them burnout and failures in care are inevitable’ Qualtrics 2020 Anonymous Nurse.
PT6 Team cohesion IF frontline team cohesion is prioritised THEN staff feel less isolated, increase sense of peer support and sense of belonging LEADING TO Staff less likely to leave/turnover reduced.	Summary: Team relationships and cohesion leads to staff less likely to leave/turnover reduced. References: Drennan et al 2015, Efendi et al 2018 Stakeholders example quote: ‘Feeling valued and respected as part of your team’ Qualtrics 2020 Anonymous AHP

Phase II findings: characteristics of included studies from structured searches

In total 3277 titles and abstracts were screened leading to full text review of 383 articles and final identification of 88 relevant papers across all six PTs, with many included papers being relevant to more than one PT. The main exclusion reasons at this stage were lack of identifiable results relating to mental health staff, lack of qualitative findings and not relevant to the UK context. Only three full texts were excluded due to being unobtainable. Numbers of full text articles included for each PT were as follows: Workload: 26; Quality of Care: 37; Team Cohesion: 41; Leadership: 13; Development: 36; Supervision: 27 (categories are not mutually exclusive). The Preferred Reporting Items for Systematic Reviews and Meta-Analyses diagram in figure 1 shows numbers of papers identified for each search and for final inclusion.

Figure 1.

Preferred Reporting Items for Systematic Reviews and Meta-Analyses diagram of included studies. NHS, National Health Service.mh= mental health, QOC = quality of care

The large majority (71, 81%) of papers had been published since 2010, with the remainder (17, 19%) between 2004 and 2009. Papers predominantly reported studies in England (65, 74%), with another 13 (15%) covering more than one nation or the UK as a whole. Very few related specifically to the other nations (Scotland 2, Wales 2, Northern Ireland 1). The predominant data collection methods were interviews (54, 61%), surveys or questionnaires (21, 24%) or focus groups (18, 20%), with some studies employing more than one approach. Study size varied widely depending on the method used, with some surveys including several hundred people while some interview studies included less than 10 participants. Participants were often drawn from a variety of professions (38, 43%); where they focused on a single group, nurses were most common (26, 30%). Eight studies included patient views. Where the setting was specified, studies were approximately evenly split between inpatient and community. Details in relation to all the above characteristics were not always reported, but was only missing for a few studies except for study setting (33, 38%).

A full list of included papers mapped to their respective PTs is presented in online supplemental appendix table 3.

Phase III findings: final PTs and logic model

The three core PTs finally identified were interconnectedness of workload and quality of care, investment in staff support and development and involvement of staff and service users in development of policies and practice. Figure 2 presents these PTs within the final overarching logic model developed to offer a holistic picture of what is required to encourage best practice for retention of the mental health workforce.

Figure 2.

The logic model of factors affecting the retention of mental health staff.

Revised programme theory

From the initial PTs in phase I the hypothesis was that if interventions were focused on specific contexts (such as supervision, quality of care, team leadership) it would result in improving retention within the mental health workforce. However, while refining the PTs it became apparent that many of these, we had labelled contexts, were actually occurring further downstream as mechanisms and outcomes, proceeded by organisational contexts fundamental to change.

Interconnectedness of workload and quality of care

In the early stages of the review, workload and the quality of care were identified as related, but ultimately different, concepts and assigned to separate initial PTs. As the review progressed and the wider literature was explored, we found that the two concepts were inseparable and were two aspects of the same critical driver of retention. When workload is high, due to acuity and staff shortages, staff feel they have to deliver lower quality care in order to be able to manage extended demands on their time. In relation to workload, the included studies considered not only the number of qualified staff on the ward or in community mental health services but also the experience level of those staff and if they were regular or temporary such as bank or agency staff. If the organisation did not prioritise strategies to manage these aspects of workload, then mechanisms such as perception of delivering high-quality care, ability to make a difference and therapeutic relationships were negatively affected. As most health professionals are motivated, at least in part, by a desire to help their patients, the realisation that working conditions are having a detrimental effect on their ability to make a difference leads to reduced job satisfaction and in the longer term if it continues, reduced staff morale and intention to leave.

Table 4 contains the final statements and supporting evidence.

Table 4.

Phase IIII: refined ‘IF… THEN…LEADING TO…’ statements with evidence

Brief title	Full If Then Leading
Interconnectedness of workload and quality of care	IF the organisation’s leadership prioritises safe staffing levels to meet service user acuity and numbers (including a suitable balance of permanent and temporary staff on wards, capped caseloads in the community, and an appropriate mix of staff professions and experience) (context). THEN clinical staff perceive they have a manageable workload to enable them to deliver high-quality care, build therapeutic relationships and make a difference to service users (mechanisms). This LEADS TO increased job satisfaction, improved safety and morale (for staff and service users); reduced stress and burnout and increased staff retention (outcomes).
Investment in staff support and development	IF the organisation’s leadership invests in the support and development of clinical staff (including protected time for quality supervision and regular supportive team meetings; and time and funding to undertake professional development) (context). THEN staff feel confident that they have the appropriate skills to provide high-quality care; have appropriate support to reflect on practice and process emotions; and believe they are valued members of the team and organisation (mechanisms). This LEADS TO improved clinical practice, better team relationships and a collaborative approach to service users’ care; increased job satisfaction, improved morale; reduced stress and burnout and increased staff retention (outcomes).
Involvement of staff and service users in policies and practice	IF the organisation’s leadership develops policies and practices which are informed by and involve clinical staff and service users in decisions about the delivery of services and prioritise targets relating to service user care and staff well-being (context). THEN staff feel listened to and valued, and perceive that the organisation’s values align with their own (ie, focusing on clinical need rather than targets and financial considerations) (mechanisms). This LEADS TO increased job satisfaction, improved morale; reduced stress and burnout and increased staff retention (outcomes).

Investment in staff support and development

Where organisations have developed effective structures for staff support such as supervision and offer training and development opportunities, staff felt more valued and confident in their clinical skills and subsequent quality of care provided. In phase I, we considered that different elements of investment in staff, such as high-quality supervision and training and development opportunities, constituted separate PTs. Subsequent review of the wider literature led us to conclude that these are all elements of the same organisational contextual factor. An organisation that invests in its staff, to ensure that they are supported, enriched and feel valued, is likely to also invest in clinical supervision and training and development opportunities. The level of investment and the priority given to it therefore provides a key context, which then triggers mechanisms relating to staff feeling valued and supported and more able to deliver high-quality care. Outcomes may be as diverse as impacts on patient care (better trained staff deliver better care) and staff morale, both of which lead to greater job satisfaction and improved retention.

Involvement of staff and service users in development of policies and practice

The processes of organisational policy development were not initially identified as a PT in phase I, however closer exploration of the literature revealed that these are key and underpin many other activities. If policies and practices of an organisation are developed using input from frontline staff and patients then they will be perceived to be directly relevant to what is important to those groups, they will feel included, valued and supported. In contrast, in situations such as where there is a perceived clash of values, for example when management are perceived as being very focused on financially driven targets rather than patient or staff well-being, these triggers mechanisms relating to not feeling valued, and a lack of consistency between their own values and those of the organisation. This leads to poorer job satisfaction and morale, both of which directly impact on retention.

Discussion

The aim of our realist review was to identify, from the published literature and stakeholder priorities, which factors are affecting the retention of mental health staff in UK adult services, in what contexts these factors operate and for what reasons. This can potentially increase understanding to help organisations design and implement more effective, evidence-based retention strategies and interventions. The review findings enabled the development of a programme theory incorporating three key processes which influence the retention of mental health staff. First, if organisations enable safe workload/staffing levels, staff perceive that they can provide high-quality patient care. Second, if organisations invest in training, development and protected supervision, staff feel valued and more able to provide good patient care. Finally, if the organisation’s policies and practices are informed by and congruent with staff values and prioritise patient and staff well-being, staff feel valued and supported. These processes re all vital to job satisfaction and successful staff retention.

The programme theory also allows the development of hypotheses of why things might not work and in what circumstance. Much of the included literature in this review focused on interventions targeted at individuals rather than at a team or an organisational level. The lack of workforce retention interventions focusing on the interconnectedness of individual, team and organisation has previously been highlighted in reviews targeting single professions such as nursing¹⁰ and doctors.³⁰

In line with the Kings Fund 2022 statement that workforce strategies cannot be looked at in isolation, the programme theory developed in this review suggests that retention interventions which are focused on frontline individual staff stress and resilience levels are likely to have minimal success and only short-term gain.³¹ This is supported by Foster et al³² who found that while individual retention interventions were positively received by frontline staff they led only to short-term gains and highlighted the need for organisational issues to be addressed.³² Therefore, while interventions focusing on individuals or teams may have some effect on job satisfaction, to be effective in the longer-term healthcare workforce retention strategies must be mapped back through the IF, THEN, LEADING to statements and refocused further ‘upstream’.

This review suggests organisational culture is the foundation for successful healthcare workforce retention and the key context for any mechanisms to be activated or not.

If this is the case, then significant change in retention levels requires organisational culture to be the primary focus of positive change. Previous literature has identified that interventions that address organisational and professional issues simultaneously are more likely to be successful³⁰; our review would go one step further to suggest that organisational change is a necessary precondition for positive and effective change downstream. This potentially has significant implications for healthcare organisation decisions on where to prioritise their workforce retention interventions.

Job satisfaction has been well documented as a strong predictor of turnover intent.^{33 34} This paper employed job satisfaction as a proxy to retention, its position in the final programme theory as a director precursor outcome to staff burnout, sickness and finally to staff retention. All three ‘If–Then–Leading to’ statements tracked through job satisfaction, suggesting it may be a useful measure for measuring effectiveness of any retention interventions.

Strengths and limitations

Using a realist approach is considered a strength as realist synthesis privileges relevant, rich and rigorous literature with the focus on depth and breadth of understanding. An added strength was the inclusion of job satisfaction in the search terms as a proxy for retention, which widened the search strategies to enable a more comprehensive search of relevant literature. However, the team acknowledged that the priority for synthesis was interpretation rather than ensuring the most comprehensive identification of information and is possible that other relevant evidence may have been missed or overlooked.

Another strength of the study is the embedding of stakeholders within phase I to aid development, prioritisation, refining and testing of the initial emerging PTs. The inclusion of diverse healthcare personnel, service users and academics ensured that the widest range of differing perspectives as possible was taken into account within the review process.

Although the mental health workforce is highly diverse in profession and setting, much of the included research was conducted in inpatient settings, and the resulting theory is therefore mostly based on this context. Despite the review only including mental health settings, and most of that being inpatient, the findings are potentially generalisable to other settings. For example, staffing is a retention issue in many healthcare settings. However, staffing issues in mental health contexts bring specific challenges that may not exist elsewhere, such as increased risk of violence, aggression, and self-harm.

The focus on UK NHS staff experience helped to reduce the volume of potentially relevant literature and provided a specific context for the findings but may also have limited international generalisability. However, the exploration of underlying mechanisms, rather than specific interventions, helps to increase the potential transferability of the findings. Healthcare staff retention is a complex phenomenon and other causal relationships may be present in the data and more influential in other contexts (including different countries and different healthcare settings).

Future research and recommendations

Research is needed to test the programme theory, looking at how upstream organisational changes may have a knock-on effect allowing a multiple of mechanisms to be fired in to action and looking at ways to measure the downstream effect such as job satisfaction as a proxy for staff retention. The reviewed highlighted significantly less published literature on leaderships impact on job satisfaction and retention than the other five IPTs despite the high weighting stakeholders gave this IPT.

Additionally, all the included studies were conducted in the UK, it is not known if this programme theory is applicable to both low and high resource settings. The PTs require further testing against data from low-income and middle-income countries.

Research papers looked at the opinions and satisfaction of staff who were still employed within their respective trust. No studies included in the review focussed on those staff who had already left and why.

Conclusions

This realist review facilitated the construction of robust, evidence-based and stakeholder informed PTs about the mechanisms underlying mental health workforce retention. These explanations hold the potential to support the future development and delivery of effective retention strategies and interventions. The logic model illustrates the interconnectivity of the three overarching theories. Findings highlight the importance of organisation context as preceding and activating mechanisms to produce particular retention outcomes. The findings further suggest that workforce retention interventions may need to first focus on organisational policy changes which can influence context to achieve an effect downstream.

Retention interventions which focus on ‘outcomes’ such as team cohesion may have some influence on outcomes. However, much of the evidence explored workload and staff development, so often the product of organisational decision-making. Frontline reconfigurations and individual-level interventions cannot help in the long term if organisations do not demonstrate to their staff that they are valued and supported to make a difference to patients. ‘Demonstrate' is key here, if staff do not feel valued and supported, then changes will be resisted.

Data availability statement

Data sharing not applicable as no datasets generated and/or analysed for this study.

Ethics statements

Patient consent for publication

Not applicable.

Ethics approval

Not applicable.