Abstract
Evidence based medicine (EBM) involves the integration of the best scientific evidence available with physician experience and patient preferences. One of the first steps in EBM involves developing a question based on issues encountered with patients or a disease and finding evidence to answer the question. This paper is a guide to plastic surgeons in how to begin the practice of EBM by developing clinical questions and finding evidence to answer these questions.
Keywords: evidence-based medicine, outcomes, research, questions
Clinical Scenario
You are a surgeon and your patient is a 25-year-old man with a severe type III open tibial fracture as a result of a motorcycle accident. The options for treatment are reconstruction or amputation. Understandably this is an emotional issue for the patient and he wants the leg to be saved using reconstructive surgery. Your preference is for flap reconstruction of the limb. Patient and physician preferences must be taken into account when making the decision whether to reconstruct the leg or amputate, but you question whether the decision is based on the best available scientific evidence.
In another scenario, you are a surgeon who is interested in researching outcomes in rheumatoid arthritis patients who elect to have arthroplasty to replace deformed joints in their hands. You have performed arthroplasty on numerous patients and find your patients to be very satisfied with how their hands look after surgery and report good outcomes. However, you question if their hands actually function better after surgery. You speculate that patients are reporting better outcomes because their fingers are less deformed and the improved aesthetic appearance prompts the patients to report being satisfied. You are interested in researching this topic to apply evidence to your clinical experience, but are not sure how to start.
In both cases, you have identified a gap in your knowledge, and need to identify research evidence to fill that gap. You could answer your clinical questions by reading an expert opinion article, or consulting a colleague. However, the current environment of evidence based medicine (EBM) stresses the need to develop a good question and finding high quality research to answer clinical questions. Sackett, one of the founding fathers of EBM, states that “studies show that busy clinicians who devote their scarce reading time to selective, efficient, patient driven searching, appraisal, and incorporation of the best available evidence can practice evidence based medicine”.1 As part of the emphasis to introduce the concepts and practices of EBM to readers of Plastic and Reconstructive Surgery, we present this paper as a guide to developing a clinical question and how to find the best available evidence to answer the question.
Types of clinical questions
Before developing a specific question, it is important to think about the type of question you have. Clinical questions can be narrowed down to four types: therapy, prognosis, diagnosis and economic/decision analysis. For example, if you are comparing two treatments such as amputation and reconstructive surgery, this is a therapy question. If you are interested in understanding the outcomes for older patients after breast reconstruction, this is a prognosis question. A diagnosis question might inquire about the accuracy of a new diagnostic test such as the use of ultrasound to detect silicone breast implant ruptures. If you are asking if a treatment results in lower costs and improved quality of life, this is an economic/decision analysis question.
The type of question also determines the type of study that you should look for when searching for evidence. For example, in the scenario for arthroplasty, you may want to determine if arthroplasty on the joints in the hand results in better outcomes than no surgery on the affected joints. A randomized controlled trial (RCT) is the preferred study type for answering treatment questions. However due to feasibility and ethical concerns, many surgical procedures cannot be randomly allocated or blinded. In this case, prospective cohort studies would be a better choice. When questioning the prognosis for a disease, a prospective cohort study would be more appropriate. Patients are enrolled at the time of diagnosis or treatment and followed over time to determine if any factors are associated with prognostic outcomes. Diagnostic studies that evaluate new diagnostic tests are more likely to be cross-sectional or prospective cohort studies. Economic and decision analysis questions arise when the best available evidence concerning treatments is inconclusive and an RCT is not ethically practical or feasible.
Developing a question using PICO
The next step is developing a question. You want an answer to your question but you also want to avoid spending precious free time aimlessly searching through journals. Investing a few moments to craft a question can help to narrow the search for answers. (Figure 1) The question must be clinically relevant, well thought out and answerable. For example, a treatment (e.g. anticoagulant treatment for deep venous thrombosis) or surgery (e.g. replanting an amputated thumb) that is widely accepted with outstanding results would not be a good subject for a clinical question. On the other hand, researching whether the use of costly tissue sealant can decrease the incidence of postoperative seroma is an important study question. Or whether a particular face lift technique is associated with better outcomes and less complication rate is a timely study question.
Figure 1.
PICO is a tool to help develop focused clinical questions.2-4 PICO is an acronym for Patient/Problem, Intervention, Comparison Intervention and Outcomes. The question will be more focused if you can be specific for each element. The first element in the question is a description of the patient population or problem of interest. The second and third elements contain the intervention and comparison intervention of interest respectively. An intervention is often a treatment but could also be a prognostic factor or a diagnostic test. In some cases, there will not be a comparison intervention. Lastly the question should consider the outcomes from the intervention. For example, will the intervention lead to fewer complications and less morbidity?
Using our scenarios as examples, we identify the pieces that will be used to formulate a question. In the case of tibial fractures, you consider the most severe types because these types of fractures are more likely to be considered for amputation. Based on Gustilo's classification system,5 you decide to restrict your patient population to type IIIB and IIIC due to the lack of soft tissue coverage of the fracture that requires flap reconstruction. The intervention of interest in your question is limb salvage using reconstructive surgery. This surgical procedure will then be compared to amputation. You consider what outcomes you would like to compare between these two treatments. After reviewing several of your cases, you decide to concentrate on improved limb function and quality of life after surgery. Putting the four elements together we construct a focused question: Do patients with severe type IIIB and IIIC open tibial fractures have better limb function and quality of life than patients who have amputations?
In the arthroplasty scenario, you have decided to concentrate your research efforts on rheumatoid arthritis patients who elect to replace joints in their hands. Based on your personal experience with rheumatoid hand deformities, you further decide to concentrate on arthroplasty at the metacarpophalangeal (MCP) joint. After considering the different type of implants that can be used for MCP arthroplasty, you select silicone metacarpophalangeal arthroplasty (SMPA) because it is has been used for over 40 years and is not experimental. You consider alternate surgeries but decide to compare SMPA patients with those who do not have surgery. The outcome that you decide upon is improvement in hand function as shown in patient reported questionnaires and biomechanical measurements. We put the pieces of the question together: “Do rheumatoid arthritis patients who undergo SMPA have better functional outcomes and quality of life than patients who do not have SMPA or any other type of arthroplasty”?
Hierarchy of Evidence
Before searching for evidence, you will need to consider the hierarchy of evidence. (Figure 2) The hierarchy of evidence ranks study types from those with the highest evidence to lowest evidence. RCTs, systematic reviews and meta-analyses are at the top (highest evidence) and expert opinions and case reports at the bottom (lowest evidence). Systematic reviews and meta-analyses combine or pool data from multiple individual studies using systematic, predetermined methods. This pooling of data, if done correctly, allows the reader an unbiased summary of the available research. A systematic review of RCTs is considered higher in the hierarchy than a systematic review of observational studies. A narrative review or expert opinion is also a summary of studies but is done in a non-systematic manner and therefore more likely to be biased.
Figure 2. Hierarchy of Evidence.
Electronic Databases (MEDLINE, EMBASE, CINAHL, Cochrane)
The next step in practicing EBM is finding the best available literature to answer your question. This is accomplished by searching through electronic databases such as MEDLINE, EMBASE, CINAHL and the Cochrane Library.
The Cochrane Collaboration was founded in 1993 and produces the Cochrane Library.6 The library consists of several EBM databases. Most notable is the Cochrane Database of Systematic Reviews, which consists of systematic reviews and meta-analyses. The library is updated on a quarterly basis and considered a reliable source of EBM. All systematic reviews must meet rigorous standards before publication. Because not every condition has been investigated with a systematic review, it is often necessary to look at other databases.
MEDLINE is maintained by the National Library of Medicine, contains over 11 million citations from journals dating back to 1950 and is updated weekly.7 Articles are referenced using medical subject headings (MeSH). PubMed has over 19 million citations from life science journals and MEDLINE. PubMed has several search options that can help simplify your literature search. You can search specific journals or a single citation. Another search option is called Clinical Queries, which allows the user to narrow their search by subject and clinical study category (e.g. therapy, diagnosis, prognosis) or limit the search to systematic reviews. EMBASE contains biomedical and pharmacological articles from 1947 to the present.8 It contains more than 19 million records from more than 7000 journals. Although EMBASE and MEDLINE overlap in their coverage of the literature, EMBASE has a better coverage of European journals.9 CINAHL (Cumulative Index to Nursing &Allied Health Literature) covers 2960 journals and other citations related to nursing and allied health from 1981 to present.10 CINAHL has over 2 million records.
The important thing when searching for evidence is to start at the top of the hierarchy and work your way down. You should begin by searching for systematic reviews or meta-analyses. The Cochrane Library is an excellent source for this type of evidence.6 Another choice is the American College of Physicians (ACP) Journal Club which pre-appraises evidence for physicians.11 If there is no systematic review or meta-analysis, the next step involves searching MEDLINE or one of the other databases for RCTs. Many questions are not appropriately answered by RCTs and therefore few if any RCTs exist. This is often the case for surgical procedures.12, 13 It will then be necessary to move down the hierarchy and gather evidence from prospective cohort studies, case-control studies and lastly case reports. Case reports are best for generating hypotheses. The reader needs to be aware of the lower quality of these types of studies and take that into account when evaluating the evidence.
Search terms and limits/filters
Although there may be abundant research on your question, it is necessary for expediency to restrict your search such that you have a manageable number of papers to read. When asking a clinical question that needs a timely answer, a few high quality papers are adequate. However, in the research setting, it is important to find all the relevant papers. Developing accurate search terms will aid in finding the evidence you need to answer your question. The search terms need to be specific enough to narrow your search but not so narrow so that you miss important articles. In MEDLINE, each article is assigned Medical Subject Headings (MeSH). MeSH terms can then be used to search for pertinent research. MEDLINE gives the option to explode on terms (to include more narrowly defined terms) to gather the most research. In addition to MeSH terms, it is sometimes helpful to include text words to gather evidence. MeSH terms are usually combined with Boolean operators (AND, OR) to maximize the number of articles selected. It may take several attempts using different combinations of search terms to get the most comprehensive and relevant list of articles.
All databases give you the option to put limits on your search. For example, you can limit your search by language (e.g. English), age (e.g. adults >18), humans, or type of article (e.g. review). If your search results in more than 200 articles, you may need to refine your search terms to reduce the number of articles. Medical librarians have a great deal of experience with developing search terms and it may be useful to consult one.
Once you have your list of possible articles, you can scan the titles and abstracts to determine if the article is relevant to your search. You can access the articles electronically or have them pulled and copied if necessary. If you find an article of particular interest in MEDLINE or PubMed, you have the option to find related articles that may not have been identified by your search.
Using the tibial fracture question as an example, we search MEDLINE using PubMed and select Clinical Queries as a tool. We select the systematic review option and use the following MESH terms: Tibial Fractures or Lower Leg Injuries. We further limit our search to humans and English language. A total of 176 articles are selected based on these search items. This is a reasonable number of articles and searching titles and abstracts should reduce the number further. An alternate search would be to access MEDLINE (in this case using the interface Ovid) and enter the terms (exp Tibial Fractures or Lower Leg Injuries) and (exp Amputation or exp Limb Salvage or exp Reconstructive Surgical Procedures). The exp in the search denotes that these terms were exploded. A total of 1659 articles were selected based on the search terms and limiting the search to English language, humans, and review articles brought the number down to 97 articles. After reviewing the titles and abstracts, we found a systematic review comparing the outcomes for amputation or reconstructive surgery after severe tibial fractures.14 The systematic review paper did not find any difference in outcomes for the two treatments.
In the research setting, the search items should be broad enough to find as many articles as possible. Using the scenario for arthroplasty in RA patients, we decide to search the Cochrane Library first to see if there are any systematic reviews of this topic. We enter the terms (Rheumatoid Arthritis and Arthroplasty). The search yielded 25 systematic reviews. A quick search of the titles and abstracts of the articles selected by our search did not find any relevant review articles. Many of the review articles selected for this search concern knee replacement surgery. We would like to find more relevant research on this topic and therefore, restrict the search to surgery on the metacarpophalangeal joint. Using MEDLINE, we enter the terms (exp Rheumatoid Arthritis and (expArthroplasty/ or expArthroplasty, Replacement, Finger)) and (exp Metacarpophalangeal Joint/Surgery). The search results in 112 articles, which drop to 99 when we limit articles to English. Again we search the titles and abstracts, and identify a systematic review.15 This systematic review is based mostly on retrospective studies and found improvement in hand function after SMPA. However, none of the clinical trials or prospective studies included in the review compared surgical cases to nonsurgical cases. We decide to widen our search using the same search terms but using the Boolean operator OR instead of AND in the search. For example, we change the search to: (exp Rheumatoid Arthritis AND (expArthroplasty/ OR expArthroplasty, Replacement, Finger)) OR (exp Metacarpophalangeal Joint/Surgery). This results in considerably more articles (2811). Restricting articles to the English Language drops the number of articles to 2322. Although this combination of search terms results in papers on other joints such as the knee and shoulder, we do find one prospective cohort study of SMPA patients that compared surgical cases to nonsurgical cases.16 The paper along with the systematic review has the information available to help answer your question. Additionally, both articles provide a high level of evidence.
Reporting literature searches (STARLITE)
Although you may not be publishing the results of your literature search, it is important to know the elements needed for a good search. Standards for reporting literature searches were developed with the mnemonic STARLITE (sampling strategy, type of study, approach, range of years, limits, inclusion and exclusion criteria, terms, and electronic sources).17 Standards have been developed for reporting RCTs (CONSORT - Consolidated Standards of Reporting Trials18), meta-analysis of RCTs (QUOROM - Quality of Reports of Meta-analysis19) and meta-analysis of observational studies (MOOSE - Meta-analysis of Observational Studies in Epidemiology20). In a research paper, it is important to report the elements of the literature search so that your results can be reproduced.
STARLITE was developed for qualitative studies but could also be useful when searching for quantitative studies. (Figure 3). The sampling strategy should state whether the researchers attempted to find all the available research or restricted the search in some format. The types of studies included in the search such as RCTs or prospective cohorts should be reported. This allows others to know the level of evidence that was included in your search. The approach states how you found the research (e.g. electronic data bases or other methods such as hand searching). The range of years, search limits or filters, inclusion and exclusion criteria, and terms used in the search should also be reported. If you used electronic databases they need to be listed.
Figure 3.
Conclusion
As is often the case, one question leads to another. For example, in the tibia fracture question, the best available evidence found no difference in outcomes between amputation and reconstructive surgery. You could continue your practice of performing reconstructive surgery, confident that you are providing a good outcome to your patient by saving the limb. You may still question if there are any other studies that can distinguish the outcomes for the two treatments. For example, economically, does limb reconstruction have better outcomes than amputation? If searching for evidence provides no answer, you could propose your own research to answer the question.
This paper presents the initial steps for those interested in pursuing EBM. We have provided guidance and examples to help with developing a focused clinical question and finding the highest level of research to answer the question. It may be difficult to start the process but it is a skill that improves with practice and persistence. EBM is a new direction in American medicine21 and will be used for quality improvement efforts in the US22 By applying EBM principles to integrate with physician experiences, we should be able to provide better care for our patients by eliminating idiosyncrasies in our treatments.
Acknowledgments
Supported in part by a Midcareer Investigator Award in Patient-Oriented Research (K24 AR053120) from the National Institute of Arthritis and Musculoskeletal and Skin Diseases (to Dr. Kevin C. Chung).
References
- 1.Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn't. BMJ. 1996;312:71–2. doi: 10.1136/bmj.312.7023.71. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Richardson WS, Wilson MC, Nishikawa J, Hayward RS. The well-built clinical question: a key to evidence-based decisions. ACP J Club. 1995;123:A12–3. [PubMed] [Google Scholar]
- 3.Schardt C, Adams MB, Owens T, Keitz S, Fontelo P. Utilization of the PICO framework to improve searching PubMed for clinical questions. BMC Med Inform Decis Mak. 2007;7:16. doi: 10.1186/1472-6947-7-16. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Centre for Evidence Based Medicine. 2009 http://www.cebm.net.
- 5.Gustilo RB, Mendoza RM, Williams DN. Problems in the management of type III (severe) open fractures: a new classification of type III open fractures. J Trauma. 1984;24:42–6. doi: 10.1097/00005373-198408000-00009. [DOI] [PubMed] [Google Scholar]
- 6.Cochrane Library. 2009 http://www.cochrane.org.
- 7.National Library of Medicine. MEDLINE Fact Sheet. 2009 http://nlm.nih.gov/pubs/factsheet/medline.html.
- 8.STN Database Summary Sheet EMBASE. 2009 http://www.cas.org/ONLINE/DBSS/embasess.html.
- 9.Egger M, Davey Smith G. Principles of and procedures for systematic reviews. 2nd. London: BMJ Publishing Group; 2001. [Google Scholar]
- 10.CINAHL Information Systems. 2009 http://www.cinahl.com.
- 11.American College of Physicians Journal Club. 2009 http://www.acpjc.org.
- 12.Chung KC, Burns PB, Davis Sears E. Outcomes research in hand surgery: where have we been and where should we go? J Hand Surg. 2006;31A:1373–9. doi: 10.1016/j.jhsa.2006.06.012. [DOI] [PubMed] [Google Scholar]
- 13.Chung KC, Burns PB. A guide to planning and executing a surgical randomized controlled trial. J Hand Surg. 2008;33A:407–12. doi: 10.1016/j.jhsa.2007.11.027. [DOI] [PubMed] [Google Scholar]
- 14.Saddawi-Konefka D, Kim HM, Chung KC. A systematic review of outcomes and complications of reconstruction and amputation for type IIIB and IIIC fractures of the tibia. Plast Reconstr Surg. 2008;122:1796–805. doi: 10.1097/PRS.0b013e31818d69c3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Chung KC, Kowalski CP, Myra Kim H, Kazmers IS. Patient outcomes following Swanson silastic metacarpophalangeal joint arthroplasty in the rheumatoid hand: a systematic overview. J Rheumatol. 2000;27:1395–402. [PubMed] [Google Scholar]
- 16.Chung KC, Burns PB, Wilgis EF, Burke FD, Regan M, Kim HM, et al. A multicenter clinical trial in rheumatoid arthritis comparing silicone metacarpophalangeal joint arthroplasty with medical treatment. J Hand Surg Am. 2009;34:815–23. doi: 10.1016/j.jhsa.2009.01.018. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Booth A. “Brimful of STARLITE”: toward standards for reporting literature searches. J Med Libr Assoc. 2006;94:421–9. e205. [PMC free article] [PubMed] [Google Scholar]
- 18.Begg C, Cho M, Eastwood S, Horton R, Moher D, Olkin I, et al. Improving the quality of reporting of randomized controlled trials. The CONSORT statement. JAMA. 1996;276:637–9. doi: 10.1001/jama.276.8.637. [DOI] [PubMed] [Google Scholar]
- 19.Moher D, Cook DJ, Eastwood S, Olkin I, Rennie D, Stroup DF. Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement. Quality of Reporting of Meta-analyses. Lancet. 1999;354:1896–900. doi: 10.1016/s0140-6736(99)04149-5. [DOI] [PubMed] [Google Scholar]
- 20.Stroup DF, Berlin JA, Morton SC, Olkin I, Williamson GD, Rennie D, et al. Meta-analysis of observational studies in epidemiology: a proposal for reporting. Meta-analysis Of Observational Studies in Epidemiology (MOOSE) group. JAMA. 2000;283:2008–12. doi: 10.1001/jama.283.15.2008. [DOI] [PubMed] [Google Scholar]
- 21.Chung KC, Ram AN. Evidence-based medicine: the fourth revolution in American medicine? Plast Reconstr Surg. 2009;123:389–98. doi: 10.1097/PRS.0b013e3181934742. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Chung KC, Rohrich RJ. Measuring quality of surgical care: is it attainable? Plast Reconstr Surg. 2009;123:741–49. doi: 10.1097/PRS.0b013e3181958ee2. [DOI] [PMC free article] [PubMed] [Google Scholar]