The gold standard of management? Evidence-based management and healthcare delivery

Key messages

• There is a push towards evidence-based management (EBMgt) by some academics, particularly for the management of healthcare organisations, which draws heavily from the paradigm of evidence-based medicine (EBM).

• I suggest that an evidence-based approach is not an end in itself for improving healthcare and organisational processes.

• Organisational research in primary care settings demonstrates that professional experience and values exert a strong influence on the use of evidence in practice, and perceptions of what constitutes valid evidence vary between occupational groups.

Why this matters to me

Born into a family of frontline public sector practitioners, I developed an interest during my undergraduate studies about how performance management could result in perverse or unintended consequences as well as driving improvements for patients: for example, how meeting a 4-hour A&E target might lead to patients not being screened for MRSA before being admitted to hospital wards. I began to perceive healthcare as a complex system in which performance management systems, although necessary and important, could give a distorted picture of the work difficulties and decisions facing frontline staff. I am presently researching how management knowledge is translated into healthcare organisations, and how a top-down reform agenda impacts upon what knowledge is viewed as valuable.

Abstract

This article presents ideas from a current debate in the field of Management Studies, which suggests that healthcare organisations should take an evidence-based approach to their management practice. Such arguments form part of a broader move toward the standardisation of knowledge and practice in healthcare organisations, and an overriding concern with organisational efficiency. However the complexity of healthcare decision-making and evidence-selection make this process problematic, including in primary care.

Introduction

How strategic and operational decisions are made in institutions has been a traditional focus for organisation and management studies. This paper discusses ideas in management thinking concerned with poor decision-making by managers and executives and the suggestion that organisations should practice evidence-based management (EBMgt). Does the apparent success of an evidence-based medicine (EBM) paradigm which aims to incorporate the best contemporary research evidence into professional assessments really offer a ‘gold standard’ for management practice? Or should we first look to recent research on the use of clinical evidence, and how primary care practitioners experienced it, before espousing such an approach?

Evidence-based medicine and the rise of evidence-based management

Evidence-based medicine rose to prominence in the 1990s and can be understood as a movement that sought to improve clinical outcomes across healthcare organisations by standardising professional decision-making. If decisions were based upon the most recent and validated ‘best’ evidence, then variation in practice would be stymied. Given the sheer breadth and volume of new medical knowledge entering the field, the case was apparent for developing systematic reviews and online databases that synthesised clinical knowledge and enabled medical professionals to access the latest research evidence promptly. The establishment of the Cochrane Collaboration (1993), the Centre for Reviews and Dissemination (1994), NICE (1999) and NHS Evidence (2009) addresses this need. An underlying feature of all these organisations is the belief that ready access to high quality research evidence enables professionals to make better-informed decisions. In this way, EBM works to fulfil the objective of closing the gap between ‘accumulated medical knowledge and daily clinical decisions’.¹

From the late 1990s onwards, analysts began to consider applying an evidence-based approach to the management of organisations, and healthcare settings in particular. Kovner and Rundall noted the predisposition of large healthcare organisations to rely on external management consultants for their strategic decision-making and implementation plans, and a failure by healthcare managers to ‘rigorously challenge the information upon which such recommendations are based’.² Walshe and Rundall cited the ‘overuse’ of mergers by organisations as one area of poorly informed managerial decision-making,³ adding support to the idea of using management ‘science’ to inform the design of healthcare institutions.⁴ These arguments have contributed to calls for EBMgt to be adopted by businesses generally, although given the complexity and scale of health services worldwide, it would appear most beneficial to the organisational challenges of this type of sector. Combining EBM with EBMgt is suggested to be a valuable means of improving healthcare outcomes and quality overall.⁵ To reference a paper by one of EBMgt's chief advocates in the US, ‘The need to implement effective healthcare organising has become as pressing as the need to implement medical breakthroughs.’⁶

UK health policy and evidence-based management

In the UK, the Coalition Government has expressed its support of evidence-based health care whilst at the same time being criticised for planning large-scale organisational changes that are not supported by the available evidence; for example, the strategy for penalising hospitals for emergency readmissions of patients.⁷

At the local level, primary care trusts may have been active in ‘evidence-based commissioning’ and contracting, referring to external evidence about their local population to support service provision plans. However, there is little information to suggest that healthcare organisations are moving towards an EBMgt strategy in the sense that the latest research evidence from Health Services Delivery, Management or Organisation Studies is consulted before implementing strategic change.

That EBMgt has not entered mainstream practice is not simply due to the newness of the concept or because research findings purportedly take 17 years to enter practice.^5–6 It is because a coherent and uniform body of management evidence does not exist presently, and certainly not in a readily accessible format comparable to the Cochrane Collaboration. In fact, the issue of whether it is even possible to create a synthesised repository of organisational and management knowledge is a moot point and a contentious matter in this debate.⁸ Management research can be equivocal and different types of evidence – for example, single descriptive accounts, multiple case studies and quantitative surveys – use different methodological approaches that are not readily commensurable. Thus, what EBMgt as a movement has failed to adequately acknowledge is that the complexity of organisational life, in which practitioners deal with the unintended consequences of policy interventions and unpredictable cases, escapes the parameters of the randomised control trials that have provided the ‘gold standard’ of EBM. Proponents of EBMgt have also paid little attention to the problems and criticisms associated with evidence-based practice.⁹

Systematic reviews based on existing evidence are widely supported by an evidence-based policy strategy, yet ‘real time’ evaluations of policy interventions are few and far between.¹⁰ Pawson and Tilley have therefore recommended that researchers and evaluators adopt a ‘realist’ approach to their work one that aims to understand the contexts in which change takes place. This includes attention to ‘the norms, values, and interrelationships’ of a given setting as well as the multiple mechanisms that shape it.¹¹ This assessment is pertinent to a complex system such as primary care, especially as it faces further structural transformation over the coming years. The implication is that for research evidence to be really useful in practice, local context must first be appreciated and understood. An EBMgt approach risks overlooking this point in order to promote the diffusion of standardised knowledge in healthcare organisations.

Learning from evidence-based medicine

Studies of the use of clinical evidence reveal that external information is often taken into account alongside professional experience and values, rather than prima facie, and assessed according to different criteria depending upon a person's training and position.^12–16 In their exploratory study of the meaning of evidence in primary care in Canada, (where genetic screening for breast cancer and hypertension were used as case studies), Beaulieu et al found that, ‘new knowledge that researchers and specialists may consider as ‘evidence’ suitable for transfer in primary care may not be considered as such by primary care practitioners’.¹² Instead, the decision to use knowledge by physicians centered on three aspects: 1 the context of practice; 2 perceptions of current knowledge in the field; 3 professional and personal values (ibid). Similarly, in a study of the implementation of coronary heart disease evidence-based guidelines in the UK, McGivern et al reported that different professional and occupational groups had varying conceptions of high-quality evidence which ‘limited the diffusion of evidence-based knowledge between professions’.¹³ They found that ‘GPs took a broader view of evidence, balancing the findings of trials against the needs of their local population’. PCT managers, on the other hand, were concerned with the risk of non-compliance associated with evidence-based guidelines (ibid). This supports previous empirical research on the translation of clinical knowledge into practice, which found that, ‘GPs were more ready to doubt the relevance of trials, taking a more holistic view of other research evidence and its relevance.’¹⁵ Finally, in a study of two general practices in England, Gabbay and May observed that clinicians infrequently accessed or used research evidence directly and were more reliant on their colleagues' experiences and interactions with other professionals to keep knowledge up-to-date.¹⁶

In this way, GPs were more likely to be informed by a trusted ‘community of practice’ than by utilising external, formal knowledge resources on a regular basis (ibid).

Such findings alert us to three points: firstly, the value of social and informal processes enable knowledge-sharing at a local level in time-pressured environments; secondly, research evidence is not judged according to one overarching criteria; thirdly, professionals factor in their experience alongside validated research knowledge.

Implications for general practice

Qualitative studies have revealed the tendency of healthcare professionals to access research findings in a secondary manner and rely on healthcare professionals and colleagues for information and new knowledge. Time restraints and access to resources remain a fundamental challenge; healthcare practitioners are unlikely to have the means to access and critically appraise primary research data and apply it in situ unless supported collaboratively and organisationally. The implication is that before healthcare organisations take onboard EBMgt, it would useful to reflect on how clinical evidence enters practice presently, how this process could be made more amenable to healthcare professionals in non-specialist settings, and what criteria are used to assess and validate ‘evidence’ in primary care.

Concluding remarks

Like EBM, EBMgt signals an attempt to challenge the attitudes of professionals be they clinician, manager or both to bring about a more standardised approach to the quality and efficiency of the healthcare that is delivered. It encourages practitioners to utilise robust, peer-reviewed sources of evidence for their decision making, rather than popularised management sources or consultants. Like EBM it conveys the idea that decision-making can be made more transparent and traceable. But it also signals a move away from professional decisions that are based upon experience and intuition since these are viewed as non-scientific, or non-rational, forms of evidence and do not fit neatly with an evidence-based paradigm.

Taking an evidence-based approach to management is not of itself a comprehensive solution to the problems facing healthcare organisations today. Firstly, it does not address how local public involvement and patient experiences can be factored into organisational decision making and planning, or how to ensure that academic research is practically applicable for end users, such as GPs and practice managers, and sensitive to local populations and contexts. Secondly, when the term ‘evidence-based’ is extended from its roots in medicine and clinical trials to become the predominant means of improving healthcare delivery, clinical and non-clinical knowledge are treated as equivalent and comparable, when clearly they are not. Evidence-based management is certainly right to challenge how important decisions are made and upon what sources of knowledge they depend, but it risks marginalising valuable professional experience in pursuit of an idealised form of ‘scientific’ management and organisational design.

Organisational and management research has much to contribute to the provision and design of health services; for example, by shedding light on complex systems, acknowledging risk and uncertainty, understanding incentive structures and detailing the actual work performed by professionals day-to-day (over and above that which is documented for performance and monitoring accounts such as QOF). And, with the prospect of GPs undertaking greater financial and managerial responsibilities for the commissioning of local services, it appears timely to reflect on how management research currently influences organisational decision making and whether it could play a greater, and more useful, role in future.

However, if this knowledge is channeled under the remit of EBMgt, it may appear to be somewhat of a Trojan horse in promoting a narrow hierarchy of organisational evidence in a ‘top down’ manner. One alternative is for organisational researchers and academics to uncover what knowledge is most useful to healthcare organisations and to work in partnership with professionals at different levels of the healthcare system. Attention to collaboration focuses on the relational and translational side of research to ensure that management scholarship considers practical problems and is timely, as well as theoretically insightful. Partnership working also helps to elucidate whether knowledge producers and knowledge users have conflicting priorities and how these might be resolved. The LJPC, for example, could serve as a forum for knowledge-sharing between academic institutions and general practice, helping to clarify the meaning of evidence in primary care and establish what types of non-clinical knowledge are useful to practicing GPs. In the words of one noteworthy writer, ‘we cannot understand the role of formal knowledge in our world without understanding the character of those that apply it.’¹⁷