Abstract
MEDLINE® retrieval using several information retrieval algorithms was characterized for relevance to point-of-care therapeutic decisions for a sample of clinical queries in family practice. Evaluation methodology is described and preliminary results are presented.
INTRODUCTION
In multiple studies of primary care medical practice, the phenomenon of decision-making in the presence of questions for which answers are potentially available in published medical literature has been documented [1, 2]. The Evidence-Based Medicine phenomenon of the past decade has stimulated substantial effort in the medical community to prepare systematic reviews to anticipate common therapeutic questions e.g. the Cochrane Collaboration, Clinical Evidence, and evidence-based clinical practice guidelines. However, there are many therapeutic decisions that must be made in circumstances in which systematic reviews of the clinical evidence are not available or are not applicable because of special circumstances of the clinician, patient, or environment. In these cases, there may be semantic characteristics of individual citations from the primary medical literature that identify those articles most useful for therapeutic decision-making.
METHODS
The Natural Language Processing group at the National Library of Medicine conducted an experiment comparing several algorithms to provide citations which may be more useful for therapeutic decision-making. We used MEDLINE citations for a sample of the Clinical Inquiries developed by the Family Practice Information Network (FPIN) and published in the Journal of Family Practice, as a “gold standard” for citations deemed to be best evidence for a family physician in answering common clinical queries from office-based practice. The process of literature review for FPIN clinical inquiries is described at www.fpin.org. The FPIN citations were inserted in a blinded comparison with the “top 40” citations produced by 3 experimental algorithms for information retrieval, and the first author, a board-certified family physician, was asked to rate all citations as: A (definitely useful in clinical decision-making for the query); B (relevant, but not sufficient to make a decision); C (irrelevant to clinical decision-making). For each decision, the clinician had the opportunity to make brief notations to support these rating.
RESULTS
Examples of MEDLINE citation titles and the corresponding ratings for the query “What is the most effective treatment for acute low back pain” follows:
“Non-steroidal anti-inflammatory drugs for low back pain” (systematic review) A
Does 48 hours' bed rest influence the outcome of acute low back pain? (small RCT) B
“A randomized, double-blind, placebo-controlled trial of sclerosing injections in patients with chronic low back pain.” (chronic LBP) C
Overall ratings for the 5 queries are:
| A | B | C | |
|---|---|---|---|
| Back pain | 22 | 8 | 3 |
| Obesity | 17 | 6 | 12 |
| Osteoporosis | 20 | 16 | 2 |
| Panic disorder | 25 | 9 | 1 |
| Warts | 23 | 7 | 3 |
DISCUSSION
Some of the characteristics used in decision-making (e.g. publication type and journal source) are readily identifiable in MEDLINE, but other characteristics, such as number of subjects, comparison of multiple therapies, placebo control, and availability of a therapy for the US practitioner community (e.g. approval by the US Food and Drug Administration, availability in a community environment) require sophisticated text processing, metadata, and inference. Interactive analysis of results of information retrieval between clinicians and computational linguists is essential to improving the precision and recall of clinically relevant MEDLINE output.
References
- 1.Covell DG, Uman GC, Manning PR. Information needs in office practice: are they being met? Ann Intern Med. 1985 Oct;103(4):596–9. doi: 10.7326/0003-4819-103-4-596. [DOI] [PubMed] [Google Scholar]
- 2.Gorman PN, Ash J, Wykoff L. Can primary care physicians' questions be answered using the medical journal literature? Bull Med Libr Assoc. 1994 Apr;82(2):140–6. [PMC free article] [PubMed] [Google Scholar]