Physicians and knowledge translation of statistics: Mind the gap

The evidence-based medicine movement, started more than two decades ago, focuses on diligently summarizing an almost overwhelming amount of health research to help clinicians make more informed decisions. Many years later, it has become apparent that evidence from meta-analyses is not well understood by, and is therefore less accessible to, clinicians. In a linked research article, Johnston and colleagues¹ report on their evaluation of clinicians’ ability to understand different statistical formats used as summary estimates from meta-analyses, in a well-conducted randomized survey of clinicians across eight countries. They concluded that all of the six statistical formats presented were poorly understood by the clinicians or were perceived as having limited usefulness.

Before addressing the problem highlighted in the linked article (i.e., of clinicians not understanding statistics well enough to facilitate knowledge synthesis), it might be helpful to consider why knowledge translation of health research is important. Morris and colleagues² examined time lags in the implementation of health research and found that studies consistently report an average of 17 years from publication of evidence to translation into clinical practice. In a widely cited study that surveyed randomly selected households in the United States, McGlynn and colleagues³ concluded that only half of Americans were receiving recommended care. As a key illustration of this translational gap, the Human Genome Project has been completed for more than 10 years and has made an important contribution to medical research, but it has had an impact on clinical care in only a few instances.⁴ These studies, along with a growing body of research on the gap between evidence and practice, have shifted a great deal of public and political attention toward health researchers and funding agencies to prioritize knowledge translation and translation science. We clearly need to address the typical time lag between knowledge production and knowledge use, but first we need to better recognize what creates this gap.

In their seminal paper on knowledge translation, Graham and colleagues⁵ described knowledge inquiry as first-generation knowledge that produces an unmanageable number of studies of variable quality and accessibility. Knowledge synthesis is seen as second-generation knowledge that addresses the inability of busy clinicians to synthesize all the evidence available on a particular disease or condition. With the wealth of research being disseminated and made accessible through electronic media, the implicit belief is that, optimally, clinical decisions should be based on well-documented and synthesized evidence. Leaving aside the many areas of health care in which there is currently insufficient evidence, there is still enough research to drive entire enterprises that provide synthesized research (e.g., the Cochrane Collaboration). The idea is that, with a correctly formatted query, evidence is more easily available to clinicians to guide their decision-making and improve health outcomes for patients. Unfortunately, national statistics show that use of synthesized evidence remains low, at a high cost to patients and the health care system.⁶

Our first reaction to such a dilemma is often to propose more education. If clinicians are not using the evidence, one assumes the problem lies with the clinicians; limited use of evidence cannot be the fault of the evidence — it must be the fault of the user. In the 25 years since the evidence-based medicine movement was established, medical programs have included training on critical appraisal and research methods, with little improvement in care.⁷ Windish and colleagues⁸ surveyed 11 residency programs and found that the average score for statistical knowledge was 41.4% among medical residents; this score increased to only 50.0% among residents with advanced degrees. The idea that more education for clinicians will solve the problem is further refuted by Johnston and colleagues.¹ On surveying more than 500 family physicians and internal medicine specialists internationally, they found no significant association between graduate training in health research methodology and correct interpretation of the statistical formats used in knowledge synthesis. So, if training does not address the problem, perhaps we should take a closer look at how we present our evidence.

Johnston and colleagues¹ conclude that we need to consider other ways to help clinicians understand the results of research, particularly the results of syntheses such as systematic reviews and meta-analyses. An increasing focus on patient-oriented research opens up interesting possibilities for making research findings more accessible to clinicians. The need to disseminate knowledge not only to clinicians but to patients and the public as well brings the barrier of technical language and jargon into focus. Statistics, probably more so than any other aspect of research, is full of concepts and technical terms that may be difficult to interpret, let alone communicate, and this presents an important barrier to knowledge use. The p value, for example, is one of the most basic tools used to define whether something is statistically significant, and yet it is probably one of the most misunderstood and misused concepts in medical research.⁹ If researchers misuse and misunderstand such basic statistical tools, how can we expect clinicians to comprehend or use the findings of meta-analyses?

We suggest two solutions. First, just as funding agencies often require that abstracts be written in plain language to be accessible to the public, perhaps something similar should be considered for communicating the results of statistical analyses in research papers. In other words, our message should be tailored to our target audience — a key factor for successful knowledge translation.⁷ Second, research with patients has shown that visual displays are far more comprehensible for complicated concepts. Visual presentation of statistical findings may be an excellent option to improve knowledge translation to clinicians.¹⁰ Johnston and colleagues¹ found that the clinicians in their study understood magnitude of effect best when presented as a risk difference. Routinely presenting the risk difference with a plain-language interpretation and supplemented with a graph might increase the perceived usefulness of the findings from meta-analyses.

Until we address this communication barrier in evidence synthesis and further explore potential solutions, we will continue to see a gap in the knowledge translation of statistics.

Key points

• Statistical interpretation is difficult for researchers as well as clinicians, and this is an important knowledge translation issue.

• Education has not been effective in improving statistical knowledge and interpretation among clinicians.

• More accessible language and visual displays may be needed to make statistical findings from meta-analyses more comprehensible to clinicians.