Has anything changed in Evidence-Based Medicine?

Abstract

The Evidence-Based Medicine (EBM) movement, undoubtably one of the most successful movements in medicine, questions dogma and “clinical authority” and combines the “best available evidence” with clinical expertise and patient values in order to provide the best care for the individual patient. Although since its inception in the 1990s its strong theoretical foundations remain unaltered, a lot has changed in its practical implementation due to the electronic explosion of information and the unprecedented COVID-19 crisis. The purpose of this article is to succinctly provide the reader with an update on the major changes in EBM, including the important most recent ones that were “fast-tracked” due to the COVID-19 challenge.

Introduction

Uncertainty has been, is, and always will be an inherent characteristic of medicine [1,2], and still, in the 1960s and 1970s, remained an integral part of the bedside medical practice and sometimes, diagnosis was based on “clinical authority”, personal observation and intuition, rather than standard scientific criteria [3]. Also, by the late 1970s, people recognized that clinical research has been rapidly expanding and therefore there is a need for its more efficient handling [4]. The term “Evidence-Based Medicine” (EBM) has its roots at McMaster University in Canada in the 1990s and is the first organized attempt to question dogma and “clinical authority” and factor in the best available evidence (BAE) into clinical judgement and patient factors to guide clinical decisions [5,6].

This EBM-movement better articulated that the medical teaching and practice needs to shift its paradigm in order to adapt to better handle and translate in an objective way the ever-growing amount of clinical research produced. Since its inception, it has developed into one of the most successful campaigns/concepts in medicine, but at the same time, it is constantly adapting an ever-changing environment. The purpose of this manuscript is to review what has changed in the practice of EBM and reveal some future directions stemming from the COVID-19 challenge.

Changes in EBM

Have we changed how we define EBM?

The two most widely quoted definitions of EBM come from its pioneer, the late Dr. David Sackett, being: “the conscientious, explicit and judicious use of current best evidence in making decisions about the care of the individual patients” [7], which was later refined to “EBM is a systematic approach to clinical problem solving which allows the integration of the best available research evidence with clinical expertise and patient values” [8]. But, for example, in the above definition, how can we define “judicious” within that definition and whatever this definition might be, it is not something “measurable”?

Therefore, not everything is well defined, and several more recent definitions of what EBM represents have emerged throughout the years, most of these are very close to the original EBM definition outlined above and loosely encompass all three aspects of the EBM “tripod”: Decisions to treat should be based on: (1) patient expectations/values; (2) physician skills/expertise and (3) BAE. A current working definition might therefore be: “EBM is the integration of the best available evidence with clinical expertise and the individual patient's values, preferences, and unique circumstances” [9]. A detailed list of those several interpretations is beyond the scope of this article, however, why are all these somewhat definitions constantly emerging? One therefore may pause the question: Has EBM changed to require new definitions? The answer is that definitions slightly change due to lack of an 100% applicable definition, there is considerable uncertainty as to whether everyone is on the same page when talking about EBM, and each definition aims at incorporating additional insight originating from the thought process of its inventor.

Changes in how the term “Evidence-based” is employed?

EBM is no longer “one thing, but encompasses several areas in medicine. There are also related “evidence” terms such as “Evidenced-based practice”, “Evidence-based policy”, “Evidence-based nursing”, “Evidence-based health care”, “comparative effectiveness research”, “health technology assessment”, “Medicina ex testimoniis”, which seem like to be “à la mode” and further confuse matters [10]. Of note, “alternatives” to EBM have been the so called “narrative-based medicine”, in which a patient's narrative (story) is a requisite in decision-making [11], and variations of the “person-centered health care” [12]. However, still, EBM is needed as both of these approaches should be based on the BAE, and therefore should be seen as complements instead of rivalries of EBM [10,13].

In addition, EBM has been expanded to include two different areas:[10, 14] “Evidence-based research” (EBR) (population-level) versus “Evidence-based practice” (EBP) (individual-level), which are clearly separate but act in concert [10]. EBR is “the use of prior research in a systematic and transparent way to inform a new study so that it is answering questions that matter in a valid, efficient, and accessible manner” and aims at eliminating redundant and/or unnecessary research which might jeopardize patient wellbeing [14]. Following the principles of the EBR, will also help reduce the “avoidable waste of research” [15], which in 2009 has been estimated to be 85% of the $100 billion of investment for research per year [16]. The so-called “metaresearch” field (i.e. research that evaluates research), has emerged to “study research” in a particular domain, diminish research waste and unethical use of patients in unnecessary research studies, optimize research funding resource allocation, and in general avoid randomness, errors and fraud in research practices and methods [14,17,18]. Although a detailed description of EBR is beyond the scope of this article, its current state, usefulness and intricacies could be found elsewhere for the readers who are interested [14,17,19,20].

Has the “best available evidence” changed?

Given the definitions of EBM stated above, at first glance why would anyone contest its intentions? After all, if “evidence” concludes that “something”, is “good”, then by not following it, this would be harming our patients. However, as alluded to earlier, the problem with those definitions is that are qualitative, more theoretical than practical and do not specify nor guide as to what is the “current best evidence”. How does a physician prioritize what is “best evidence, especially in the era of information overload? What are the tools to identify the best evidence and filter it from “junk”? How to separate quality from quantity?

To answer these questions, the EBM movement introduced the concept of “pyramid of evidence” (aka “EBM hierarchy”), which is behind the basis of the levels of evidence for research and clinical practice, and which is getting fine-tuned periodically [21]. In this system, evidence methodologies residing on the top of the pyramid are considered superior: At its summit are the meta-analyses and systematic reviews, followed by randomized controlled trials (RCTs), cohorts, case-control studies, case series and case reports, and at the very bottom expert opinion. An example of a typical pyramid is shown in Fig. 1 , however although almost all look the same at the top, the bottom may be modified for example to include some laboratory and animal research next to expert opinion, or in some other instances have a breakdown of the different types of observational studies [10]. The common theme is that whatever the variation, studies are separated into “robust” evidence (levels 1 and 2), versus “less robust” evidence below it.

Fig. 1.

The pyramid of evidence (or evidence hierarchy).

However, this stratification based on solely methodological grounds is an oversimplification and may be dangerous to the “naïve”. Certainly, a badly conducted RCT will provide bad results, and an appropriately conducted observational study may provide more robust results. Therefore, there has been a change in how this pyramid of evidence is perceived and several authors now agree that the hierarchy of evidence is basically a hierarchy of methodologies [10,22], and each methodology may produce good or bad results according to its use. In a similar manner, not all research questions may be answered by an RCT. The aim of a RCT, is to eliminate, if at all possible, the risk of a bias and confounding factors. Two groups of patients randomly selected, with one chosen to receive the investigative treatment, whereas the other is chosen to receive a placebo or the currently accepted standard treatment. These groups are homogenized as much as possible with regards to demographics and possible confounding factors, which gives a RCT good internal validity. The old saying “if all you have is a hammer, everything looks like a nail”, is very relevant to EBM: Obvious interventions cannot and should not be validated through RCT for ethical reasons; for example, the Heimlich manoeuvre to unblock the airway in a choking patient, or the use of cardioversion in a case of atrial fibrillation. Further from “the obvious”, other treatments have not been established through RCTs, but rather through observation, as for example insulin treatment for diabetes, the deleterious effects of tobacco and other historical examples [23,24]. Taking it even one step further, even results of RCTs have been overturned by smaller studies [25]. The latter is illustrated for example by the fact that several RCTs concluded that the nasal spray flu vaccine (aka the live attenuated vaccine) was superior to flu shots, but in later years some case control studies showed that the nasal spray was not effective which led its withdrawal from the flu season [25]. Therefore, a new, revised form of the evidence pyramid has been suggested highlighting the overlap in study quality (Fig. 2 ) [26]. In this paradigm, straight lines have been replaced by wavy lines, illustrating the overlap in study quality between different levels of evidence. In addition, the top level of the traditional pyramid pertaining to systematic reviews and meta-analyses, has been “chopped off” and replaced by a lens through which evidence is evaluated.

Fig. 2.

The “new” pyramid of evidence [26].

In a similar manner, the GRADE system (Grades of Recommendation Assessment, Development and Evaluation) [27] has been developed in which traditionally “less robust” levels of evidence may be upgraded and “more robust” levels may be downgraded according to quality of evidence, however since hierarchies are by default inflexible, every attempt with tackling these, share the problems outlined above [10,28].

What has changed in identifying the BAE?

In the past, there was less medical literature, however it was a daunting logistical task to assemble it together fast due to the lack of electronic resources. Nowadays, information is at our fingertips, but the electronic revolution has stimulated an unprecedented explosion of the amount of information available, equally if not more daunting when trying to answer a particular clinical question. To mitigate this, the EBM Working Group had originally recommended a 4S model for sorting out this problem which later was upgraded to a 6S model[29], [30], [31]. As shown in Fig. 3 , the search for an answer to a clinical question, must be first begin in the top layers and then proceed down to the next layer if that fails. “Systems” refers to computerized clinical decision support systems (CDSS), which are linked to the electronic health record of a particular patient and into which data for that particular patient would be automatically inserted into computer algorithms coupled to important relevant research data which is used to generate patient-specific recommendations to the physicians [32]. If available data is provided there, the physician needs to look no further, however the problem is that these are not widely available yet [29,31]. Next layer is the “summaries” layer which for example may include a clinical pathway, concise textbook summaries, or clinical practice guidelines (CPG). Examples may include databases such as “Up to date” (www.updtodate.com), Dynamed (www.ebscohost.com/dynamed/default.php), etc. Interestingly, acute care hospitals with access to Up to date versus the ones that did not, had significantly lower complication, mortality rate and hospital stay [33]. The next layer is “synopsis of syntheses”, which are comprehensive concise answers to a particular clinical query based on high-quality evidence (eg. Systematic reviews) which can be found for example in evidence-based abstraction journals, such as the British Medical Journal (ebm.bmj.com) or the Database of Abstracts of Reviews of Effects (DARE) (https://www.york.ac.uk/crd/). The “syntheses” are usually systematic reviews, including the Cochrane library (www.thecochranelibrary.com), and if there is none, then the next stage would be to look at “synopses of single studies”. These are pre-appraised, brief summaries of a single study again found in evidence-based abstraction journals. The final stop would be a single original study. If all these measures fail, then a more traditional approach would be to search in non-appraised evidence databases, for example in PubMed (www.pubmed.org), Embase (www.embase.com), etc. Detailed lists of all the resources is outside the scope of this paper and the reader may consult the article by Windish for further information [31].

Fig. 3.

The EBM Working Group 6S system [29].

Has the clinical implementation of EBM changed?

Frequently and contrary to its definitions, EBM is mistaken to be synonymous with only the “medical knowledge” aspect of the practice (i.e. the “evidence”) leaving out the physician and patient. The concept of “shared decision making”, which is one of the key aspects of patient-centered care, further enhances evidence-based practice[34]. Shared understanding is the first step which also helps establish a partnership between patient and clinician through a process of “collaborative deliberation” which concludes when the best course of action for that particular patient has been reached[34]. Shared decision making is particularly pertinent when the evidence backing up the different options is of lower quality, being less than ideal, and if happens, then the patient's expectations, values and circumstances may weigh towards the physician's suggestion[34]. For these reasons, patient decision aids (PtDAs) have been created, and defined as “interventions that support patients by making their decisions explicit, providing information about options and associated benefits/harms, and helping clarify congruence between decisions and personal values” [35]. They are not patient education tools for a particular condition; they rather include a concise but precise presentation of the current (pre-appraised) high-quality evidence in a way that can be readily communicated so that patients can understand it, have a clear picture about what matters to them, what to expect, and become more involved in the decision-making [35]. Despite the fact that their benefits have been shown [36], their implementation is considerably lacking behind in some disciplines, for example in orthopaedic surgery, with only 7 studies in a 2008 review [37].

Has the teaching of EBM changed?

The ability to appropriately identifying and critically appraising the evidence is a sine qua non skill for the effective practice of EBM. As with every skill, it takes time and effort to develop it and teaching is essential and can be acquired at any stage of one's career [38]. Therefore, several initiatives as for example the European Union Evidence-Based Medicine project [39], the teaching programs of the Oxford Centre for EBM and McMaster University to disseminate EBM teaching, the uptake of teaching resources is low [40,41]. Several strategies both face-to-face and virtual have being implemented to more effectively teach EBM including journal clubs, lectures, workshops, group work, seminars, collaboration with librarians and newer methods as well including simulations, gaming and the use of mobile phones [40]. Sadly, the bedside practice of EBM by is “irregular” at best, and even those “trained”, do not exhibit a change of behaviour giving a reason of “lack of skills” [42]. Talking about change, over 300 articles have been published on the teaching of EBM [43], and despite the vast amount of literature, the teaching of EBM remains problematic [[40], [41], [42], 44]. There is ongoing discussion and debate with no universal consensus regarding the best approach as the acquisition of skills in EBM is fairly complex, related to contextual factors, needs clinical integration and positive role models [42]. It looks like the change from theory to effective practice in the teaching of EBM has not been accomplished.

The COVID-19 “stress-test”–How EBM has changed?

The ongoing COVID-19 pandemic has provided an unprecedented challenge for EBM, in both the research and clinical application level [45], [46], [47], [48]. Why? Because per EBM, the most robust evidence comes from RCTs and meta-analyses, followed by less robust evidence according to the evidence pyramids described above. Although the number of patients eligible to participate in studies, as well as the number of publications is increasing at a staggering level [46], is that commensurate with the quality of research? That seems not to be the case as the majority of the 3,000 COVID-19-related registered clinical trials have either very small numbers or suffer severe methodological flaws or constitute duplicate research [48].

One may argue that what was needed was urgent research, fast-track results, urgently applied in a setting of acute public health threat where time equals lives and when decisions have to be made on a moment-to-moment basis to serve public safety [49]. RCTs cannot provide answers in this setting, and the usual “red-tape” must be quickly set aside in order to obtain high quality evidence fast. On the other hand, the combination of uncertainty with urgency should not allow erroneous/methodologically unsound research to spread faster than the disease through media, such as in the case of hydroxychloroquine [50], and other “miraculous” treatments.

This is where proper using and not dangerously “misusing” traditional EBM requires practitioners to have the essential skills to understand, critically appraise, implement and evaluate the new findings, however this is becoming challenging as alluded to earlier [51]. As far as EBM, therefore, should there be a change to shift from RCTs towards well conducted but “less robust” (i.e. observational) studies? Hopefully not, and there have been good examples of trials that have been exemplary (quality and speed) such as the trials for vaccines or the RECOVERY trial performed in the U.K [48]. Therefore, in such extreme circumstances, well-coordinated, joined, large-scale national and international efforts should aim at producing high-quality evidence and minimize research waste [48]. Of note, the Cochrane collaboration has significantly cut down its median turnover time for review production from 2 years to 3-6 months [48]. Large organizations, such as professional bodies and governments sought to provide urgent “syntheses of evidences” during the pandemic, in a desperate effort to appraise the huge amount of information, filter the junk and provide high-quality insight on what is really happening [48]. Examples include the COVID-19 Living Overview of Evidence (L-OVE), COVID-END (COVID-19 Evidence Network to support Decision-making), the COVID-NMA initiative, etc. [48].

Lastly, “common sense” should not be sacrificed in anticipation of “bullet-proof” RCTs. Obvious interventions cannot and should not be validated through RCT for ethical reasons; for example, the Heimlich manoeuvre to unblock the airway in a choking patient, or the use of cardioversion in a case of atrial fibrillation. Similarly, during the pandemic, and despite the presence of “less robust” evidence, widespread mask use was significantly delayed with deleterious consequences. Therefore, the COVID-19 crisis lesson is that although the theoretical principles of EBM remain sound, their practical implementation has rapidly changed.

Conclusion

During the past three decades, EBM has revolutionized the way medicine is practiced. Its theoretical fundamental tenet which includes management using the BAE, the patient's values and expectations and physician's skills and expertise remains unaltered. However, as presented herein, its practical implementation in the world of information explosion has significantly changed including definitions, newer expansions of the term EBM, better ways in identifying the BAE, improvements in research and clinical implementation, and EBM teaching. None of these are perfect, and the recent COVID-19 pandemic was an opportunity to identify several weaknesses, but also to accelerate many positive changes. The future will still be challenging for as EBM, and as the world is now more rapidly changing than ever before, EBM will have to catch up with the ever-growing demands for higher level patient care.

CRediT authorship contribution statement

GDC: Conceived and designed the paper; Wrote paper ADP: Wrote the paper PVG: Wrote the paper; provided critical comments and supervised the paper.

Funding

None.

Declaration of Competing Interest

None.