Doctors' knowledge about evidence based medicine terminology

In a report published in this issue (p 950) Australian general practitioners rated themselves and were then tested on their evidence based medicine skills.¹ The results are not encouraging. Fifty general practitioners in Australia rated their understanding of seven common terms from evidence based medicine from “It would not be helpful for me to understand this term” to “I understand this and could explain it to others.” On average, only 22% said they understood each term and could explain it to others. Worse still, in the subsequent structured interview only one general practitioner could provide a fully satisfactory explanation of any of the terms, and many of the explanations revealed considerable misunderstanding. The authors of the study argue that general practitioners need to understand these terms to practise evidence based medicine and that there is little good research on how this can be done. For those working in evidence based medicine these results make depressing reading.

There are some problems interpreting this study. The authors attempt to validate self rating of evidence based medicine skills, but what they actually test is knowledge. The authors recognise that people who cannot demonstrate knowledge in a potentially intimidating academic environment may be more successful at using knowledge in real life. The ability to explain a term may not be the kind of knowledge required of general practitioners. The criteria for fully understanding each term were also quite challenging. It is possible to explain a term without providing all the stated criteria (for example the criteria for number needed to treat includes mentioning that it is the reciprocal of the absolute risk reduction). Unfortunately, even if the results from those general practitioners whose answers were partially correct are combined with the fully correct answers, we are still left with poor results. Only four of 74 claims of understanding were confirmed or partially confirmed in the subsequent interview. Many, whilst claiming understanding, refused to explain the terms, which may or may not indicate ignorance. We do not know how the sample was selected or how many general practitioners had had formal training in critical appraisal. Can we apply the results from this group of Australian general practitioners to clinicians around the world? Australia is seen as a centre for evidence based medicine and one would not expect much better results elsewhere.²

The results cast doubt on the use of self assessed knowledge as a proxy for actual skills. This supports earlier research in the United Kingdom that examined knowledge of six evidence based concepts, two of which were used in the Australian study (relative risk and absolute risk).³ Khan et al studied 55 healthcare professionals including some hospital doctors. They found poor correlation between participants' self evaluated knowledge and multiple choice test scores.

These studies support the view that evidence based medicine skills are not well developed in general practitioners around the world. They do not tell us about general practitioners' demand for evidence and certainly do not undermine the case for providing it. In a survey of English general practitioners most felt that their role is in the application of evidence based conclusions.⁴ Only a small minority felt that their time was best used learning the skills of evidence based medicine. A recent trend has been to distinguish users of evidence from searchers and appraisers of evidence.⁵ The assumptions are that the skills required of users of evidence are primarily those of relating the evidence to particular patients and qualitative explanation of the risks and benefits of treatment options, rather than mastery of clinical epidemiology. However, even users of evidence may need to communicate with patients who have done their own search, and who need to weigh conflicting evidence. Although there is disagreement on the best method of teaching critical appraisal, there is evidence that a variety of methods can improve knowledge.⁶,⁷ What we do not know is which skills of users of evidence are necessary to improve consultations or patient outcomes.

The challenge for those working in evidence based medicine is to provide summaries of the evidence in a variety of formats that reflect the range of skills of users of evidence, using innovative methods of presentation. These should be arranged hierarchically so that those with interest and skills can drill down to find detail. This transparency is the best safeguard to ensure against bias in pre-appraised summaries. More and better training may not be amiss either.

Primary care p 950