Tools for Identifying Reliable Evidence and Implementing it in Everyday Clinical Care

Abstract

Just as translational medicine follows a long winding path from bench-to-bedside, so can Evidence-Based Medicine be envisioned as comprising a multi-step pipeline, from building evidence from raw data through synthesizing best practices and providing clinical decision support in a process described as the “evidence pyramid”. 1 At one end, a heterogeneous mix of clinical and experimental studies including clinical trials, case reports, animal models and retrospective analyses are published as new knowledge. Then, experts collect and assess high-quality relevant evidence on specific issues and publish their conclusions (e.g., regarding efficacy and safety of treatments) as systematic reviews and meta-analyses. Finally, when an expert consensus has been reached, this must reach the attention of policy makers within the profession, the government and insurance companies, resulting in new practice guidelines and altered clinical practice within hospitals and clinics. At each stage, this process requires a large investment of time and effort from many individuals with a wide range of expertise.

Our panel will discuss the variety of innovative approaches that are being taken by different informatics research groups to improve each step within the evidence based medicine pipeline. These approaches are, in part, devoted to making existing data collection and synthesis practices faster and more efficient, but they also involve re-imagining and re-engineering the processes by which evidence is accumulated, evaluated and applied.

General Description

The practice of Evidence-Based Medicine (EBM) is critically dependent upon judicious use of the best and most up-to-date evidence to patient care decision making. 2 Few clinical practitioners would disagree that this fundamental orientation is useful and appropriate. However, actually putting EBM into practice has uncovered many challenges. For example: How do we identify the medical topics for which evidence review and synthesis are most needed? Current groups such as the Cochrane Collaboration currently produce systematic reviews for only about 8,000 topics; this is not nearly enough to cover common and important medical conditions. 3 For a given medical topic, how can all of the relevant literature be appropriately and efficiently collected, reviewed, and synthesized? What counts as evidence appropriate for a particular medical question? Do randomized controlled trials (RCTs) always provide the best quality evidence? What about medical questions for which RCTs do not exist or would be considered unethical? Finally, how can we best ensure that EBM reports impact clinical guidelines and electronic decision support tools to improve the care given by medical, dental, nursing and allied health practitioners?

These challenges can be roughly grouped according to the steps in the process of transforming individual units of evidence (such as publications) into a form that is effective for guiding and improving medical practice:

1. Identifying topics requiring an analysis or re-analysis of the available evidence.

2. Gathering a comprehensive set of evidence that is relevant to the topic.

3. Assessing/synthesizing the available evidence in a timely and efficient manner.

4. Making high quality evidence syntheses easily available.

5. Translating evidence syntheses into action through practice guidelines and clinical decision support tools.

This process is cyclic; as medical research progresses and additional knowledge is discovered, new areas are identified that require evidence and analysis, and areas previously reviewed require updating. Therefore, the effort required to meet the EBM goal of “judicious use of the best available evidence” is large, ongoing, and exponentially increasing. 4 Currently the steps in this process are implemented in a manner that is incomplete, cumbersome, loosely organized, and largely manual. The entire process could be improved by appropriate informatics tools that support the individual activities as well as the transitions or “hand offs” of the output of one step to the next.

This panel will discuss current and potential future work in applying advanced informatics tools and platforms to facilitate the EBM pipeline. Speakers will summarize the current state of work in their focused areas of research, and will lead a discussion on the opportunities and challenges for creating an integrated, public, collaborative platform of tools that will speed up the process of creating, disseminating, and applying the best available evidence across all areas of medicine and realize the full potential of EBM.

Panelists and Presentations

Aaron M. Cohen, MD, MS is a leader in the application of machine learning techniques to biomedical literature and clinical text. Dr. Cohen has published extensively on the topics of automated clinical and bibliographic text processing, EBM and systematic review, and the intersection between these subjects. He and Dr. Smalheiser are dual PIs of the NIH-funded Text Mining Pipeline for Evidence-Based Medicine project. He will discuss the issues, challenges, and current progress in building a text mining based pipeline specifically to meet the needs of researchers performing systematic reviews across all domains of medicine, supporting steps 1–3 detailed above. This pipeline focuses on the step of identifying the relevant and appropriate evidence to include within a systematic review.

Dina Demner-Fushman, MD, PhD is an expert in the area of automatic clinical question answering and multimodal biomedical information retrieval. The clinical question answering system developed by Dr. Demner-Fushman is based on the EBM domain model and uses the EBM framework of a well-formed clinical question. She will discuss automatic approaches to gathering and synthesizing evidence in order to answer a clinician’s questions that arise during healthcare practice, supporting steps 3–5 as described above. Including, the issues in automating the initial triage of the automatically retrieved documents, the limitations of the well-formed question framework with respect to fine-grained question answering, and future work in extracting, ranking and summarizing evidence for question answering. She will illustrate practical details of implementing steps 3–5 using the NLM InfoBot system that provides an Evidence Based Practice dashboard for the Inter-Disciplinary teams at the NIH Clinical Center.

Alfonso Iorio, MD, is an internist with a research interest in strategies to address the gap between knowledge generation and its utilization in everyday clinical practice. He is member of the McMaster University Health Information Research Unit. Dr. Iorio will discuss PLUS, a curated database of methodologically sound and clinically relevant papers, and its current and potential applications. In particular, he will review strategies to integrate in a sustainable way PLUS evidence into online textbooks, clinical decision support systems, clinical practice guidelines and systematic reviews with the objective of maintaining these resources up-to-date., He will mainly focus on step 4, and secondarily steps 3 and 5 of the process described above. Finally, he will illustrate how PLUS has been integrated in a “one stop shopping” federated search engine to answer clinical questions at the point of care.

Ida Sim, MD, PhD is an expert in the area of knowledge representation and open approaches for clinical research and evidence-based practice. She leads the Ontology of Clinical Research (OCRe) team, as well as the Human Studies Database Project, which is federating computable descriptions of interventional and observational studies using OCRe as the reference semantics. Dr. Sim co-founded OpenmHealth, a nonprofit organization to create an open software architecture that will include shared modules for generating individual and population-level evidence on the effectiveness of mobile health interventions. The current EBM cycle — which spans years from initiation of research, to synthesis in systematic reviews, to application of evidence to practice — is too slow and is wholly inadequate for the needs of health care and health promotion. Dr. Sim will discuss opportunities for using ontology-based tools to support a much tighter and rapid link between evidence generation and usage, providing a semantic backbone supporting steps 1–5 in the general description.

Neil R. Smalheiser, MD, PhD has directed multi-disciplinary, multi-institutional research consortia dedicated to text mining, knowledge discovery, and bioinformatics, which have created a suite of theoretical models, databases, open source software, and free, public web-based services. He will moderate the panel, lead the Q&A, and facilitate a broad, wide ranging discussion on the major theme of the panel, which is how informatics tools can best support the EBM processes of transforming data into evidence and then evidence into healthcare practice.

Statement

All listed persons have participated in the creation of this proposal and have agreed to participate in the panel presentation.