Redefining leadership within the NHS’ complex adaptive system

Abstract

The NHS is continuously evolving and with it, traditional notions of leadership and management must be reimagined and redefined. In order to be effective leaders, however, we must first gain a deeper understanding of the context in which we lead and recognise how to navigate the system's intricacies. This article explores the characteristics of Complex Adaptive Systems and how we can understand the patterns of these systems through our experiences leading in the NHS. It then analyses leadership approaches both past and present and considers how future leaders can be effective in tomorrow's NHS. Finally, this article identifies potential barriers and challenges that leaders must overcome as they adapt and tailor their leadership approach to meet the needs of a dynamic health service.

The dynamic NHS leader

The modern NHS is an ever-changing entity, morphing between iterations in response to resource constraints, demographic shifts and advancements in healthcare knowledge. In this way, the NHS is truly dynamic; ebbing and flowing with the tide of legislative priorities and in the continuous pursuit of delivering outstanding healthcare. Whilst our services may be responsive, it is possible that leaders may continue to use outdated leadership approaches, resulting in missed opportunities for improving our services. A dynamic NHS calls for dynamic leaders; leaders who are able to adapt their approaches to address the prevailing challenges, respond to changing demands and meet the presenting need. The dynamic NHS leader is not married to a certain style or approach, burdened by the notions of a ‘traditional’ leader; they are responsive, shifting and morphing their leadership to most effectively navigate the issue of the day, with an acute awareness of the system's characteristics and constraints. In gaining a deeper understanding of the complex system in which we lead, we can recognise the approaches which are not fit for the future and be at the forefront of changing culture and attitudes towards leadership. The purpose of this article is to describe the NHS’ complex adaptive system, developing a clearer picture of the system's landscape and help NHS leaders gain clarity on the system in which they lead. It will critically analyse leadership approaches from the past and future, helping leaders to understand where skills may be most effective and examine how we can effectively prepare the next generation of NHS leaders. It will identify barriers to achieving a high-performing, well-led system, and how we may be able to start overcoming these challenges.

Characteristics of a complex adaptive system

The NHS is driven by its people and their interactions with each other, the prescribed work processes and the contextual environment. Within a complex adaptive system (CAS), we can characterise these people as ‘agents;' that is it say, someone who exerts influence to produce a particular effect or change.¹ We can better understand how the NHS operates through the CAS lens: it is a network of interacting and interdependent agents, undertaking a complex function, with the capacity for adaptation and learning from past experiences.² Further defining characteristics of CAS³ are evident throughout the NHS; it is a non-linear and dynamic system, resulting in system behaviours that can seem chaotic. Independent and intelligent agents drive the system's functions, with conflicting needs, desires and goals. This conflict in perceived goals can arise where the objectives between leadership and frontline staff are misaligned. Strategy and policy can clash with a clinical focus; assurance and regulation conflict with autonomy and independence; financial prudence can see friction where additional resources are required.

Within a CAS, control is dispersed, with no single point of command, and behaviours can emerge as a result of adaptation and learning, for better and for worse. This can be understood as the process of ‘self-organisation,' where a group's actions align into a single focus without centralised arrangement.⁴ The first responsibility of a leader within this system is to understand what drives the system, how to create the conditions for creativity and positive cultures to flourish and how agents can be influenced in order for the system to produce positive outcomes.

Our historic approach to analysing adverse incidents indicates that the NHS has previously been understood as a complicated system with elements of simple systems thinking. Root Cause Analysis (RCA)⁵ was long advocated as the preferred investigation method, focusing on identifying simple cause and effect relationships between actions and outcomes, relying on the investigator being able to deconstruct care delivery into simple steps. Complicated systems feature a large number of actions being taken within clearly defined parameters, policies and guidelines, however these actions can be mastered and perfected with professionals’ expertise, increasing the predictability of the outcomes.⁶ Surgeries, for example, are complicated procedures that can separated into stages and experienced surgeons can achieve mastery of this process with practice. By analysing healthcare outcomes and adverse incidents from a CAS perspective, we see that people have agency, meaning that they have autonomy over their actions and the ability to exert power and influence. They are also influenced by external pressures and human factors, which aims to understand human behaviours and their interaction with procedures, processes and the physical environment around them, alongside considering their limitations and capabilities. These may include the interface between people and their workplace systems, how fatigue and stress is managed and how resilient the personnel are to strained service function. Interacting components within the system and agents’ response to these can lead to emergent behaviours, observed both in positive clinical practice and in adverse incidents. Examining a negative outcome in surgery from a simple or complicated system perspective, the root cause may be that the surgeon has not followed the process. Analysing the same incident from a CAS perspective may identify that it was caused by a team culture that lacks psychological safety, inadequate safety controls, political influences from non-clinical managers or training deficits.

Leading people

Complex systems are shaped largely by interactive relationships, and it is key for leaders to facilitate structures in which these relationships can function effectively. Recognising CAS as a network of agents working towards a common goal, the first responsibility of a leader is to identify, communicate and maintain that shared vision. The vision must be relevant, easy to understand and connect with the shared ethos of the team. This creates intrinsic motivation among team members, aligning the organisations’ vision with agents’ sense of purpose. When this happens, the CAS can function through self-organisation. This results in the emergence of behaviours within the system that are more likely to be result in positive outcomes.

It is essential for leaders to better understand what motivates the people who work within systems. In many organisations, leaders can look simply to increasing remuneration due to its simplicity, ignoring more nuanced factors which may require significant changes to the way in which the organisation functions. Ariely⁷ and Gurerk's research⁸ demonstrated that financial incentives can result in a negative impact on overall performance, particularly when cognitive skill and flexibility is required. Instead, leaders must understand what gives individuals purpose and meaning in their roles, such as structure and guidance, freedom and space, data and resources or trust and autonomy.

Uhl-Bien's Complexity Leadership Theory (CLT)² is a framework for leadership within CAS. It defines leadership as a dynamic process between people in which someone is influenced by another.⁹ CLT recognises that context is a key determinant in how this dynamic process is expressed and understood, meaning that leaders must adapt their approach depending on the context in which they lead. CLT also highlights that ‘leaders’ emerge through the dynamic process of influencing others, regardless of role. This is challenged by the current structure of the NHS, with accountability and responsibility being designated by seniority, with the more senior leaders often in management roles and holding greater influence on the overall direction of the system. Collaboration between leaders and managers is key; when high social friction arises between these two positions, organisational progress can be affected. Innovation relies on low social friction and high intellection friction, as a result of diversity of viewpoint and perspective.¹⁰

The Dutch ‘Buurtzorg Model'¹¹ provides a case study of decentralised leadership within a healthcare context, where local areas self-govern their services without a ‘central command', and evidence demonstrates the effectiveness of this approach. The devolution of responsibility to Integrated Care Boards (ICBs) points to a beginning of decentralisation within NHS leadership, allowing local areas to tailor their budget allocation to meet the needs of their local population. This could be taken a step further with increased distribution of leadership within ICBs and NHS Trusts, limiting hierarchy and striving for self-organisation, with individual agents afforded autonomy and purpose.¹² Leadership in this system could be considered a ‘team sport'¹³ with the functions of a leader held between a number of individuals. In practice, this would make leadership across the NHS more sustainable, as accountability would be shared and people would feel invested in creating change. This repositioning of the traditional hierarchy would be a valuable step in moving towards a reimagined leadership approach within the NHS.

Future leaders

In preparing leaders for the future of the NHS, we must first be clear on what types of leaders are required and how they can be effectively developed. Fraser and Greenhalgh¹⁴ distinguish competence (the persons’ knowledge, skills and attributes), from capability (their ability to adapt to change, knowledge and continually improve performance). Leadership development has increasingly focussed on the development of competence however effective leadership within CAS requires adaptability, with leaders able to move between different approaches depending on the context. This may include shifting from abstract thinking when conceptualising a transformation idea to a practical perspective when implementing that idea.

Realigning our approach to developing future leaders, we should first examine the values and personal qualities leaders require in order to be effective in the NHS. Collins’ Level 5 Leadership¹⁵ identifies prevailing qualities in effective leaders such as humility, modesty and diligence. There are an abundance of people with these qualities in the NHS, and we should seek to develop their capability; that is, helping them to gain greater capacity, emotional maturity and cognitive flexibility to navigate challenges.¹⁶ The Messenger Report¹⁷ further reinforces this focus on leadership development centred on a set of unified values and purposes. A values-based leadership development approach highlights the importance of appointing the right people, allocating them to the roles that best utilise their skills and fostering an environment for them to use their skills to drive organisational change. As Collins¹⁵ writes, ‘put your best people on your biggest opportunities, not your biggest problems.'

Challenges and barriers

It is essential for services to remain contemporary in their provision of care, learning from adverse incidents and prioritising continuous quality improvement. Those leading these processes must understand how the CAS functions, recognising that patient safety incidents should direct us to make improvements to the system, rather than apportion blame to individuals. The NHS Patient Safety Strategy,¹⁸ moving towards a systems-based approach to patient safety investigations, is an important step in this direction. Stretton's Lilypond Model¹⁹ also advocates this approach, analysing the ‘ecosystem’ of an organisation or team in order to gain a deeper understanding of why outcomes occur and what improvements can be made to address these. Quality improvement is an iterative process of mapping priorities, planning tests of change, evaluating results and refining change ideas. Leaders must be mindful however that these iterations do not add to the complexity of the system or make it more difficult to navigate; changes in NHS policy often come as add-ons to existing structures, and over time, the system can become convoluted and cause confusion. Refinement of the system is vital, which includes identifying and removing aspects that do not add value or clarity to its function.

A barrier to achieving high-functioning CAS is the tendency for poor-performing systems, particularly large ones such as the NHS, to default to conservatism and cautiousness, stifling innovation. To address this, we must prioritise the principles of psychological safety, which can be described as fostering an environment in which people feel empowered to share ideas, solutions and perspectives without fear of recrimination or judgement.²⁰ Psychological safety challenges cautiousness as it promotes democratised innovation and shared growth, reflecting and celebrating positive outcomes whilst learning from negative outcomes.

The frustrating truth is that NHS leaders will continue to wrestle with constraints imposed on the system beyond our control. The ethos of the NHS is apolitical, though the political landscape is often the primary determinant of the direction of travel for the health service. NHS funding and resource provision is determined by the Treasury, with national policy and priorities set out by the Department of Health and Social Care and NHS England. Societal attitudes and expectations can also dictate the narrative, as we saw during and after the COVID-19 pandemic, with public outpouring of support for NHS staff driving the debate around remuneration and the value of healthcare workers.

Whilst these challenges may be beyond our remit, responsive and dynamic leaders are those that recognise the changing winds of the system and adapt their approaches to meet the times. In this way, dynamic leadership does not correlate with personal charisma or gravitational character. Rather, dynamism calls for flexibility, the ability of a leader to morph and shape their approach to the challenge in front of them, employing a selective set of skills based on the assessment of the context. Dynamic leadership within the modern NHS is therefore driven by thoughtful analysis, considered decision making and a deep awareness of interrelating factors at work within the system.

Expanding the evidence base

There is emerging empirical evidence demonstrating the effectiveness of Complexity Leadership approaches, characterised by the dynamic situational leadership described in this piece, where the leader adapts their approach dependent on the presenting content. Cureton²¹ was the first to identify a statistically significant relationship between effective complexity leadership and desired outcomes, in this case support for innovation, improved team learning and efficacy. Subsequent work by Newton–Lewis, Munar and Chanturidze²² highlighted that carefully calibrated leadership interventions based on the presenting context leads improved performance management within a complex adaptive health systems. Expansion of the evidence base, particular in the post-pandemic NHS, would be valuable in providing NHS leaders with further guidance for their development.