Clever searching for evidence

Do people who write about and propagate evidence based medicine use its principles properly? A paper in this week's BMJ and two other recent BMJ papers report on such authors' abilities to find the best evidence.^1-3 On the way, they have provided excellent search strategies, using filters they call “hedges” (as in hedging one's bets) that help to separate the wheat (scientifically strong studies of diagnosis, treatment, and systematic reviews) from the chaff (less rigorous ones) in one of the most frequently accessed medical literature databases, Medline.

Why is this important? We still often need to search large databases such as Medline to find original research data because reviews may not cover our questions, may be out of date, and may not be relevant enough to real clinical problems. Databases of primary research are staggeringly large (there are more than 12 million citations in Medline, and 7 million in Embase). Most research papers are written as communications from scientist to scientist and relatively few have immediate clinical relevance. Most of the remainder are not rigorous enough to warrant applying clinically.⁴ Subsequently, the proportion of useful information is very small.^w1

This lack of high quality, clinically relevant research studies leads to insecurity about the results of literature searches (table). Finding evidence can often seem easy, but those searching may either be blissfully unaware that they have not found the best evidence, or might suspect that there is still better to be found.⁵ But when a search retrieves nothing worth while, was it because there is none, or is it that the person looking just did not find it?

Table 1.

Lack of security about the results of a search

	Evidence
Search	Positive	Negative
Positive	Success	Suboptimal evidence (potentially misleading)
Negative	Uncertainty—is there really no evidence?	Uncertainty—or did we just not find it?

Skills limitations—both perceived and real—are important barriers to effective searching.⁵ Efforts have been made to simplify use of the large databases by allowing searchers to search with simple statements and minimal knowledge of the technicalities of the way the database works.^w2 But, until this approach has been found effective in pragmatic trials, we have to deal with some associated consequences. Simple searching using free text can give the impression that searching in the medical literature databases is easy and can be performed in the same manner as a general internet search. But this is wrong.^w3 We still have to know how to formulate a search in each particular database. Even the best filters^1-3 cannot compensate for a poorly structured content search strategy.

For example, in Pubmed a searcher may enter the term “Myocardial infarct^*”, thinking that the truncation will improve their search by retrieving articles using the terms myocardial infarction, myocardial infarctions, and myocardial infarcts. But the truncation (^*) removes the automatic term mapping feature, which searches for articles using both the text word and MeSH term, reducing retrieval and potentially missing articles.

Furthermore, many clinicians do not search at all⁶,^{7 w4} or look for answers in easily located but less evidence based sources.⁷,⁸ To delay clinical activity to look for the best evidence is not even considered by most doctors. There seems to be a “90 second rule”—that is, clinicians don't even try to find information unless they think they can do it in a minute and a half.⁹,¹⁰ Yet, with practice and access to certain sources, this is possible during routine ward rounds.¹¹

Searching is only one necessary skill. Whenever we run evidence based practice workshops, we are struck by the overemphasis that clinicians place on searching for and accessing evidence and their dismay at finding they also need to acquire other skills—asking an initial useful question, appraising, and applying evidence—as well.

Is using evidence in these ways too hard, and a waste of time for people who are only ever going to be amateurs, and who could refer their questions to experts? Is this analogous to pathology and medical imaging where clinicians need to know the principles of how investigations are done, but do not have to do the tests themselves?

Ideally, a literature searching service would be woven into the medical record, or made available at some point even closer to the clinical decision. Although this is still a dream for most clinicians, literature search services are available that provide evaluated and summarised evidence, and answer specific questions.¹² Efforts to nationally coordinate and develop these services are currently under way in the United Kingdom.^w5

Another alternative to clinicians searching is to use compendiums of the evidence. An increasing number of excellent resources is available that either synthesise primary research (such as the Cochrane Library), or summarise it (such as Clinical Evidence, Bandolier, TRIP Database Plus, and evidence based clinical guidelines). Valuable sources of abstracted and appraised clinically relevant primary studies (such as Best Evidence) are also available. But these abstracted services are not always easy to access, tend to focus on interventions rather than diagnosis or prognosis, are more prone to being out of date—and are now so prolific that they, in turn, have to be searched for and appraised.

Clinicians risk missing something important if they do not dip their fingers into the mess of searching for, finding, and using the best evidence, at least for some of their questions. Being able to weigh the evidence in one's hand, to first feel the heft of it, could lead to a greater ability to use it well. Using the evidence in processed form potentially loses so much in understanding issues of quality, the quantity of an effect, and generalisability. Finding the needle will then be only a part of the battle with the haystack.