Evidence-Based Laboratory Medicine: Is It Working in Practice?

Abstract

The principles of Evidence-Based Medicine have been established for about two decades, with the need for evidence-based clinical practice now being accepted in most health systems around the world. These principles can be employed in laboratory medicine. The key steps in evidence-based practice, namely (i) formulating the question; (ii) searching for evidence; (iii) appraising evidence; (iv) applying evidence; and (v) assessing the experience are all accepted but, as yet, translation into daily clinical and laboratory practice has been slow. Furthermore, the demand for evidence-based laboratory medicine (EBLM) has been slow to develop.

There are many contrasting observations about laboratory medicine, for example (i) there is too much testing vs insufficient testing; (ii) testing is expensive vs laboratories are expected to generate income; and (iii) test results have little impact on outcomes vs test results are crucial to clinical decision making. However, there is little evidence to support any of these observations. Integrating the principles of EBLM into routine practice will help to resolve some of these issues by identifying (a) where laboratory medicine fits into the care pathway; (b) where testing is appropriate; (c) the nature and quality of evidence required to demonstrate the clinical utility of a test; (d) how the test result impacts on clinical actions; (e) where changes in the care pathway will occur; and (f) where benefit/value can be achieved. These answers will help to establish the culture of EBLM in clinical and laboratory practice.

Introduction

The principles of Evidence-Based Medicine (EBM) were developed in the early 1990s¹ and slowly began to be put into practice soon after. At the beginning of the next decade the impact of the principles of EBM in laboratory medicine was recognised² and, at the same time, approaches to assessing the efficacy of diagnostic imaging on the patient pathway were being explored.³ The common theme was the emphasis on employment of the best evidence and the impact on patient outcomes. Clearly, EBM embraces the diagnostic modalities while Evidence-Based Laboratory Medicine (EBLM) focuses on the use of diagnostic tests and the goal of improving patient outcomes. A definition of EBLM, developed from the definition of EBM given by Sackett et al.⁴ is ‘the conscientious, judicious and explicit use of best evidence in the use of laboratory medicine investigations for assisting in making decisions about the care of individual patients’.⁵ So where are we today in the practice of EBLM?

One of the first definitions of EBM was ‘the integration of best research evidence with clinical expertise and patient values’,¹ another being ‘the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients’.⁴ Glasziou and colleagues suggested that a key objective of EBM was ‘....trying to improve the quality of the information on which decisions are based’, and that EBM is ‘….not about mechanisms, but about outcomes….’.⁶ These observations should resonate with laboratory professionals whilst it is recognised that the evidence base of laboratory medicine, and the use of the laboratory in the context of patient outcomes, is poor.^2,7 This poses the question as to the real impact of diagnostic testing on patient outcomes, and how the impact can be assessed⁸ and then maximised in practice.

So have the principles of EBM and the culture of Evidence-Based Practice been adopted successfully in the practice of laboratory medicine? Laboratory medicine is an integral part of the practice of medicine and so any analysis of the impact of EBLM on the practice of laboratory medicine has to look at the quality of evidence that can be used to inform clinical decision making, and the success with which this evidence is adopted. Is there sufficient understanding of the clinical process, of clinical need and clinical decision making, to enable the required evidence to be generated and to then be used effectively?

Adoption of the principles of EBLM into routine laboratory practice has been slow.⁹ Review of the literature on any potential biomarker (test) will invariably expose a rich vein of basic scientific research that establishes the relationship between the biomarker and the pathology of a disease. However, whilst this information may allude to the potential of the biomarker as a diagnostic test, it does not provide any evidence as to how the test might be used in the care of individual patients, or to improve outcomes.⁹ Sackett and Haynes suggested that a hierarchy of evidence was necessary to establish the evidence for use of a diagnostic test.¹⁰ However, whilst they focused on diagnostic performance, there are other ways in which a test may be employed, e.g. screening, treatment selection and monitoring. Furthermore, the availability of evidence, in itself, does not guarantee appropriate utilisation; robust implementation and performance strategies have to be put in place.¹¹

There may be one area of practice where this criticism does not hold, and that is the use of tests in screening programmes. The WHO criteria, developed by Wilson and Junger in 1968¹² and used in many countries where there are national screening programmes, include (i) the condition should be an important health problem; (ii) the natural history of the disease should be understood; (iii) there should be a recognisable early stage; (iv) there should be a suitable and acceptable test; (v) there should be an accepted treatment or useful intervention; (vi) treatment started at an early stage should be of more benefit than treatment started later; (vii) screening should be repeated at intervals when there is an insidious onset; (viii) the intervention should do more good than harm; (ix) adequate facilities for screening and treating abnormalities detected; and (x) the cost should be balanced against benefit. The critical point is that the test result is linked to the decision made, and action taken.

The Practical Steps in Practicing EBLM

There are five steps in the practice of EBM⁶ which have been adapted to EBLM (Figure 1).^2,5,13 Each of these steps will be briefly described below, illustrating how each step is relevant to routine laboratory practice.

Figure 1.

The EBLM cycle (adapted from ref.13).

Asking the Right Question

Picking up the observation made by Glasziou et al.,⁶ laboratory medicine is about improving ‘the quality of the information on which decisions are based’. Asking the right question identifies the unmet clinical need or problem. The effectiveness of the test (or investigation) determines the success in solving the problem and facilitates decision making. Getting the question wrong, or not asking the question at all, invalidates all of the subsequent steps and makes it likely that using the test will be of no value. Asking the question in the right way helps in the process of finding an answer.

The basic framework of the question, the PICO framework, was established several years ago. This framework identifies the Patient or population, the test (or Intervention) of interest, the procedure with which the test of interest is being compared (Comparator) and the expected Outcome.^5,6,13,14 There have been additional elements suggested, depending on the type of question. Firstly, the P can be further defined with the use of S for ‘spectrum’ or ‘setting’ as it is recognised that these are important determinants of disease prevalence and comorbidities (hence PSICO). An additional P for prior test (hence PPICO), an additional T for time (hence PICOT), and an additional C for change management (hence PICCO) have also been described.⁵ The T draws attention to the fact that timing of sample collection or duration of prognostic study periods may be important, i.e. shorter time periods may give a false impression of prognostic performance. Change management can be particularly important in outcome studies, particularly where the mode of delivery changes e.g. point-of-care testing.⁵

Responses may be required to questions posed in a wide variety of settings and the rationale for structuring the question in a particular way is to ensure that the problem is fully understood, so as to maximise the benefit from the response.¹⁵ Scenarios where questions are asked include:

• pre-analytical considerations for a test e.g. time of collection

• analytical performance of a test e.g. impact of test imprecision on outcome

• mode of delivery of a test e.g. use of point-of-care testing

• diagnostic performance of a screening test

• use of a test for diagnosis e.g. rule in or rule out

• use of a test for prognosis

• use of a test for treatment selection i.e. personalised medicine

• use of test for treatment optimisation and compliance

• health economics of a test

• test utilisation i.e. audit/performance management

• commissioning/decommissioning of tests.

These questions may be posed in a range of clinical settings and by a range of health care professionals involved in patient care, from policy makers to carers, as well as by patients. Furthermore the questions are also used as a starting point for all of the subsequent steps of the EBLM cycle.

Acquiring the Evidence

Acquiring evidence involves either finding robust evidence or developing new evidence, and so reflects both routine practice and research settings. Routine applications follow a request for more information, e.g. the scenarios outlined earlier. There are publications that describe how to search the literature,^5,6,13 including guidelines on how to undertake a quick search of Medline.¹⁶ This can be accomplished in real time, as in the case of a ward round,¹⁷ so long as the electronic library facilities are available. Google can also be a useful tool for finding information quickly. A more comprehensive search is undertaken as the first stage in preparing a systematic review, or in the development of a clinical guideline.¹⁸ In the absence of any good data arising from such searches, the next stage will be to consider the generation of new evidence through the planning of appropriate experimental studies.

There are well recognised approaches to developing the evidence for diagnostic and prognostic accuracy.^5,13 However, demonstration of the impact of a diagnostic test on health outcomes has proved to be more challenging, as it requires evaluation of the test together with the relevant intervention. Thus evaluation of the impact of a test on health outcome requires an intervention study, investigating a ‘test-and-act’ intervention where the accepted approach is to employ a randomised controlled trial. This may be considered as the ideal trial design, but Lord et al. have identified that, in certain circumstances, diagnostic accuracy may be sufficient¹⁹ and in a more recent publication have advocated the hypothetical randomised controlled trial approach.²⁰ Bossuyt et al. have also advocated the use of comparative accuracy, taking into account the current and new testing pathways.⁸ Central to these approaches is a clear understanding of the care pathway, where the test (result) fits into the pathway, and how the result will affect the pathway.

In all of these scenarios the message from the first step, i.e. getting the question right, is central to the relevance and quality of the evidence acquired.^5,13,14

Appraising the Evidence

Reference was made earlier to the need for robust evidence and the poor quality of evidence in the field of diagnostic testing. Sackett and Haynes illustrated how it is possible to draw incorrect conclusions from poor evidence.¹⁰ Critical appraisal of evidence is the key to identifying robust evidence and has been described as ‘the process of carefully and systematically examining research to judge its trustworthiness, and its value and relevance in a particular context’.²¹ Critical appraisal is ‘weighing up the evidence to assess its validity (closeness to the truth) and usefulness (clinical applicability)’.²² There are a number of critical appraisal tools that have been developed, some with specific applications for diagnostic tests. These include the Critical Appraisal Skills Program (CASP),²³ the Screening and Diagnostic Test Evaluation Program (STEP),²⁴ the Centre for Evidence Based Medicine (CEBM),²⁵ as well as specific tools for critical appraisal of papers for systematic reviews^25–27 and for clinical guidelines.²⁸ Several of the references in this paper discuss the attributes of good quality evidence and offer a hierarchy of criteria which can be used as a quick guide.

In practical terms, critical appraisal is a core skill for anyone working in laboratory medicine and can be summarised in terms of a number of simple questions when reviewing evidence, as follows:

• Is the question clear?

• Are the results of the study valid? This is called internal validity.

• What are the results?

• Are the results relevant to my patient (or population)? This is called external validity.

Applying the Evidence

Application of evidence addresses external validity but again highlights the importance of getting the question right in the first place. It is about matching the relevant evidence to the problem. It also draws together a number of strands of evidence and the detail of studies comprising the evidence. Here again, it is likely to follow the PICCO framework. Thus:

• P: Does the evidence apply to the patient (or population) and setting in the original question?

• I: Does my method have the required performance and am I using the right decision (cut-off) values?

• C: Is the current standard of care similar to the control groups featured in the evidence?

• C: What is the change in practice required and is it achievable?

• O: What were the outcomes shown in the evidence, and are they appropriate and achievable?

These high level questions identify the key elements of laboratory medicine practice, (i) does the service address relevant clinical needs (appropriateness); (ii) are the analytical and logistic quality specifications meeting the clinical need; (iii) does the laboratory service meet the established standard of care; (iv) is the process of care optimal and does it make the best use of the laboratory service, including process change, when introducing new technology or knowledge; and (v) are the best health outcomes being achieved? This approach can be applied to both established tests as well as in the introduction of new tests. However a crucial point in applying evidence is that the result has to be acted upon.

Auditing the Experience

Auditing the experience has two basics elements (i) as a learning experience and (ii) applying the principles of EBLM in daily practice, which is the starting point for good quality management and quality improvement.⁵ As a learning experience, practicing EBLM should be an integral part of the training of all pathologists and clinical scientists, as it embodies the key research, communication and management skills to work as part of a clinical team offering an integrated approach to patient centred care. In the context of introducing a new test, the audit assesses whether application of the new evidence (either found in the literature or generated from experimental studies) has led to an improved outcome. Employing audit (and thus EBLM) on a day-to-day basis is about ensuring the quality of the service, including the appropriate use of the service. The questions posed in the previous paragraph provide the link between the principles and the practice of EBLM and inform the performance metrics (or key performance indicators) that should be used when assessing the quality of the laboratory service. This is where one can make the judgement as to whether EBLM is moving from theory into practice.

Where EBLM Should Define Routine Laboratory Practice

The following are examples of the broad areas in which EBLM skills are employed in routine practice:

• responding to enquiries from clinicians and carers

  • - what test to use

  • - how to use a test

  • - the meaning of a specific result

  • - what action to take on receipt of a result

• commissioning (introducing) a new test

  • - a clinician requests a new test

  • - a manager wants a business case for a new test

  • - a manager wants an implementation programme

  • - a policy maker is considering a new screening programme

• decommissioning an old test or another part of the service

  • - identifying where new test will impact on care pathway

  • - identifying opportunities for disinvestment

• performance management of, and quality improvement in, current services

  • - managers want to audit test utilisation

  • - managers want to see evidence of predicted benefits accruing

  • - managers want to manage the introduction of a new test against specific goals

• research and development and strategic planning

  • - purchasers (insurers) are requesting improved health outcomes

  • - policy makers are considering changing the model of delivery of care

  • - leading service transformation and improvement.

Whilst many of these scenarios are often addressed reactively, if the laboratory is to play a full part in the clinical team, they should be addressed proactively; this is the key to clinical leadership in laboratory medicine. Laboratory medicine is a means of facilitating an efficient process of care and the achievement of improved health outcomes. However, the poor quality of evidence in the field of laboratory medicine has in large part been due to a lack of application of the principles of EBLM at all stages of the EBLM cycle; poor definition of the question, poor design of studies, limited critical appraisal of evidence, poor application of evidence and limited use of audit in performance management. The natural history of individual diagnostic tests tends to bear out this observation, with slow adoption epitomised by a struggle to articulate the clinical question(s).

Laboratory Medicine Services: Today and Moving Forward

What is the origin of the service that a laboratory is required to provide? It may be defined in a laboratory handbook and is typically purchased and assessed according to analytical, logistical and volume performance metrics, and on the basis of the most recent contract. However, there are a number of more strategic issues that constrain the optimal use of a laboratory medicine service, such as the following:

• It is rare that laboratory medicine is directly involved in strategic planning of health services, despite the role that the laboratory is acknowledged to play in delivering health care services.¹¹

• It is purchased, either as an independent, fee-for-service basis, or as part of a ‘tariff-based secondary care contract’.

• It is a business model that is based on cost and volume i.e. a silo budget,²⁹ rather than on value and outcomes. This is a broader problem in health care.³⁰

• The level of accountability is almost exclusively limited to financial accountability, combined with quality standards that focus almost entirely on analytical and logistical metrics.

• There is little importance given to the management role of audit, through commitment to continuous quality improvement and performance management in the context of the care pathway.

• Introduction of new tests occurs in a rather ad hoc fashion, with reimbursement in some health care systems (but not all), which may require some justification as to clinical need and evidence of effectiveness.

• There is a general recognition that there is evidence of both over-requesting³¹ as well as under-requesting.³²

• There is also a perception that the spend on laboratory medicine services is seen as excessive, as judged by the pressures put on laboratory budgets in recent years.

Against this background, there are well documented barriers to the adoption of new technology in health care, including (i) silo budgeting; (ii) poor quality of evidence; (iii) poor implementation planning; (iv) lack of prioritisation of technologies; and (v) limited decommissioning of old technologies.³³

Many of the issues can be addressed if there is a cultural change on the part of both purchaser and provider organisations in relation to the role of diagnostic services. This culture can be summarised in a few generic questions, in this case posed to the requester, on the use of a test:

• Is the test any good?

• Why do you want the test?

• What will you do with the result?

• What decision will you make?

• What action will you take?

• What outcome would you expect?

• What will be the cost?

• What benefit is expected, and where in the care pathway?

However, a fundamental problem remains and that is the way in which services are purchased and reimbursed, namely cost and volume. An alternative approach that has been described is based on commissioning of diagnostic services which involves four elements (i) identifying the (unmet) clinical need; (ii) specifying the service; (iii) contracting the service; and (iv) performance managing the service delivered. The first and last stages in this cycle represent the change from current culture and practice in laboratory medicine, namely that the service required is (transparently) based on the unmet clinical need, and whether that need is met, as measured through improved health outcomes.^34,35 Note the performance metrics are care pathway related with surrogate intermediate laboratory outcome measures. The link back to the EBLM cycle should be clear; the similarity in the activities of EBLM, commissioning and audit are illustrated in Figure 2. It should also be noted that EBLM is a tool that can be employed by those who ‘purchase’ and ‘use’ laboratory medicine services. There are broader initiatives in the reform of health care in which it will be important to evaluate the role of laboratory medicine services, how they are delivered and how they are used. These include the concepts of integrated care,³⁶ value-based health care,³⁷ and accountable care organisations,³⁸ together with addressing the barriers to innovation.³⁹ Optimal use of diagnostic services and the way in which they are delivered, should be key parts of successful health care reform.

Figure 2.

Illustrating the close synergies between the EBLM cycle (adapted from ref. 13), commissioning and performance management (adapted from ref. 36), and the process of audit (adapted from ref. 2).

Challenges of Translating EBLM into Routine Practice

There are continuing calls to improve the quality of health care by adoption of a more evidence-based approach to care, to improve efficiency and reduce errors, and to be more patient-centred in these endeavours.⁴⁰ In laboratory medicine this poses four major challenges in order for the laboratory to play an effective role in what is a major service transformation challenge, both for the health service in general, as well as for the laboratory.

The first challenge is the involvement in strategic planning which, at a practical (and local) level, requires annual review of unmet clinical needs and follows on from performance management. This will only be achieved by working more closely with policy makers, purchasers and clinicians to better understand their needs and show how the laboratory can meet these needs.

The second challenge is the generation of evidence to demonstrate that the laboratory can contribute to satisfying both the unmet needs and maximising the effectiveness and efficiency of the established, routine, clinical service. The evidence informs the business case, which must recognise the decommissioning of inappropriate tests and other services across the care pathway, if appropriate.

The third challenge is ensuring that the process of care is optimised, as well as changed, when a new test, or testing modality, is introduced. The challenge here is that this needs to be employed across the whole care pathway, not just the laboratory element. If the laboratory is to play a full part in the clinical team then ensuring results are acted upon appropriately is an important part of an efficient and effective service.

Many of these activities then come together in the fourth challenge; performance management. If laboratory services are to be used effectively, performance has to be assessed across the whole pathway, rather than simply the delivery of the right result, at the right time, for the right patient. Furthermore, under- and over-utilisation of tests can only be judged in the context of the care pathway and the health outcomes.

These are all major challenges and, in the absence of evidence, where does one begin? Accountability for quality, both clinical and fiscal, is now the focus for all health care systems and this has to be an opportunity for laboratory medicine. Some practical opportunities include:

• Build audits of test utilisation (and subsequent action) into routine quality improvement, based on the clinical question(s) being posed.

• Be aware of new clinical developments and the role that the laboratory service will play.

• Build the business case for new investigations around where benefits are to be found, and where disinvestment in other services is possible.

• Be aware of where changes in clinical practice will follow from introducing new procedures, and monitor adherence to change.

A central feature of this strategy is that the functions of laboratory medicine extend beyond the boundaries of the laboratory.

Conclusions

The principles of EBLM underpin the practice of laboratory medicine. Whilst the evidence is central to good practice, there are many challenges in deciding what evidence is required and how it is applied. The application of the principles of EBLM across the whole care pathway can help to improve quality of care, support innovation through more effective adoption of new technologies and ensure more appropriate use of health care resources. This constitutes continuous quality improvement on the part of laboratory medicine within the framework of a patient-focused care pathway. The business model for laboratory services, however, will have to change with a shift from cost and volume to value and benefits.