Putting evidence into practice: how middle and low income countries “get it together”

Short abstract

The scarcity of resources in poorer countries means that ensuring health care is evidence based is particularly important. A group of workers active in the field describe their experiences of trying to do just that

Imagine a new drug that reduces the absolute risk of treatment failure by three quarters—a rare situation in the West but a reality in countries where malaria is endemic, and where adding artesunate to existing drugs has this effect on cure.¹ In middle and low income countries, life threatening infectious diseases are everywhere: new drugs can therefore have large effects on outcomes, and even modest benefits from new interventions can have a dramatic impact on health overall. In addition, wasting resources on ineffective interventions results in technical inefficiencies and substantial opportunity costs in countries least able to afford them; the Global Fund's purchase of ineffective drugs is a recent example.²

Since 1990 there has been a massive collective effort, largely fuelled by the Cochrane Collaboration, for people from middle and low income countries to “get it together”—to work collectively to bring research evidence into systematic reviews and to consider ways to ensure the findings are used in clinical practice. We all want to put research into practice, but in the past the emphasis was implementing results from single studies. Now it is widely accepted that we need to “globalise the evidence, and localise the decision”—that is, set the results from a single study in the context of other relevant research.³,⁴ However, these syntheses of the evidence must then be actively managed to ensure change: they require dissemination, policies and systems that enable change, and influential people motivated to stimulate change. This article highlights some of our experiences and personal observations of preparing reviews and implementing change.

Shifting global bench marks

The World Health Organization has made progress in formulating evidence based policies. As a technical resource for health systems globally, WHO is central to establishing the ingredients of health care—which drugs to use and which prevention strategies work best. WHO has mandated that all its guidelines must follow an explicit, evidence based process that uses Cochrane reviews and the “GRADE” approach to guide judgments on the quality of evidence and strength of recommendations.⁵,⁶

Medical specialists in diarrhoea have been using meta-analysis for some years, and a few years ago a Cochrane review was commissioned to examine the effects of reduced osmolarity oral rehydration solution. The review extracted data from all randomised controlled trials, and the pragmatic primary outcome—the need for intravenous infusion—was clearly lower in the reduced osmolarity group.⁷ These findings were central to WHO and Unicef recommending a change to the oral rehydration salts for diarrhoea, and the new, reduced osmolarity salts are now being manufactured and distributed.⁸

Amodiaquine was widely used to treat malaria until 1988, when WHO deleted it from their “global essential drugs” list because of concerns about its safety. As drug resistance to chloroquine increased, amodiaquine was reconsidered: WHO staff prepared a Cochrane review, which showed that amodiaquine was more effective than chloroquine, with no evidence of serious adverse events.⁹ A later extensive review of safety helped to reassure the WHO Essential Drugs Committee, and amodiaquine was returned to the essential drugs list in 2003. These summaries provided an evidence base that helped persuade countries to reintroduce amodiaquine as first line treatment for malaria.

The experience with amodiaquine led to a collaboration between WHO and the Cochrane Collaboration on artemisinin derivatives. The review revealed that the research was chaotic, with many small studies, some testing comparisons that were uninformative, and showed the need for a more focused research question and agenda.¹⁰ WHO with an international group of experienced malaria researchers embarked on a series of studies of the effects of adding artesunate to existing antimalarial drug regimens, with a meta-analysis of individual patient data of over 5000 patients.¹ This has helped provide a solid base for the recommendations to change first line treatment that are currently being considered.

In reproductive health, the WHO and a network of scientists continue to identify uncertainties through systematic reviews and establish reliable research to answer key questions. They have found evidence that magnesium sulphate is not used as widely as it should be for eclampsia, despite the landmark eclampsia trial having been published almost 10 years ago and despite it being the drug of choice in WHO policy. So shifting policy is one thing, but changing clinical behaviour is another.

Figure 1.

Educational materials used in the “better births initiative” in South Africa—a low cost programme to communicate best practice to midwives and doctors on labour wards

Shifting the clinical community

It seems reasonable to communicate the principles of evidence based medicine to healthcare staff to help them use the information from systematic reviews. The principles are relatively new to many people, so we believe that engaging people in the ideas is a prerequisite to achieving changes in clinical practice based on understanding.

Policy makers participating in seminars about evidence based approaches often agree with the principles but are unsure what to do next. We developed a dissemination framework, based on work at the Centre for Reviews and Dissemination at the University of York, to guide policy makers and clinicians in understanding research summaries and in using them to change practice. The framework outlines different levels of dissemination with activities to engage different target groups; from passive dissemination, to engaging clinicians in change, to projects to highlight the potential for change, to institutionalising evidence based approaches in training and ministries (see box). We have used this framework in an international programme of research and development, the Effective Health Care Alliance, to promote evidence based practice with collaborators across the world including in China, Thailand, Nigeria, South Africa, Ghana, Tanzania, and Chile.

We experienced the many constraints facing clinicians considering practice change through the “better births initiative”—a low cost educational programme started in South Africa to communicate best practice to midwives and doctors on labour wards in low and middle income settings.¹¹ The initiative was a pilot innovation project (level 3 in the box) to improve policies and practice in labour rooms through the use of evidence from systematic reviews packaged in an interactive workshop delivered by a local opinion leader.¹² It encouraged uptake of practices known to be beneficial (such as social support in labour) and discouraged painful practices with little evidence of benefit (such as routine episiotomy).

Accompanying qualitative research in South Africa helped elucidate possible pathways for change. Any chance of change required mobilising the social structure within the health facilities, but even then change was often random—it might depend on just one person or on a team consensus to bypass the traditional staff hierarchy. Although resource constraints are often cited as a barrier to change, the reality is often more complicated: the high turnover of staff seen in one pilot site in Gauteng province, South Africa, clearly mitigated against any sustained practice change.¹³ Change on the ground is clearly a complex process, and enthusiasts seem important in the early stages: a particular respected obstetrician was the driving force for the spread of the better births initiative in Eastern and Western Cape and Kwa-Zulu Natal, with provincial government support.

In Shanghai we examined similar issues in institutional care some five years ago, when WHO and others were disseminating ideas about evidence based reproductive health. With respected leaders at Fudan University, we developed an awareness raising package (level 1 in the box) to communicate the concepts of evidence based reproductive health to a wide audience and to stimulate debate on how to use review findings in practice. An audit of practice in Shanghai maternity hospitals highlighted areas where practice was inconsistent with available evidence,¹⁴ and a repeat audit four years later showed some change in some hospitals. However, there was no policy shift at national level until recently. In the past few months, after much lobbying and advocacy, a national level training programme to apply evidence based care in obstetrics, gynaecology, and paediatrics has been planned and held. Some hospitals have rewritten their own service guidelines based on current evidence, and the Ministry of Health has agreed to use evidence based guidelines for obstetric care for basic health services in its “safe motherhood programme.”

Framework for dissemination and implementation of evidence based medicine

Level 1: Awareness raising

Purpose

• Increase awareness about effective interventions and the potential gains from using research based knowledge in policy and practice

Activities

• Produce and publish relevant systematic reviews in a variety of professional and consumer publications

• Communicate potential relevance of systematic reviews to current practice, with examples through commentaries

Level 2: Targeting groups and individuals responsible for implementation

Purpose

• Identify target groups and individuals with specific roles in implementing research based knowledge in practice

Activities

• Identify target groups, such as health ministry policy makers, donor aid advisers, professional groups, managers with responsibility for clinical and public health policy

• Communicate results from systematic reviews and their implications for practice face to face and with short summaries

• Give examples of how others have used systematic reviews combined with audit to change practice for the better in their own hospital or practice.

• Make people aware of the evidence base for effective practice change

Level 3: Pilot and innovation projects

Purpose

• Support individuals in specific pilot projects to evaluate potential ways to implement change in practices that seem to run contrary to current available evidence

Activities

• Identify collaborators engaged in or interested in developing pilot projects to implement research findings and where they perceive there is an opportunity to make care more evidence based

• Help them in to stimulate change (such as by audit and feedback or by means of opinion leaders) to practices for which there is reliable evidence from systematic reviews of effectiveness

• Ensure collaborators monitor change in policy and practice

Level 4: National or institutional policies for evidence based decisions

Purpose

• Encourage national governments, institutions, or donors to commit to evidence based approaches, with effective implementation and monitoring systems

Activities

• Work with government and donors in establishing or strengthening health technology assessment offices or similar bodies at national level

• Encourage national policies for evidence based guidelines, with management systems to ensure that guidelines are implemented and monitored

• Help institutions to train doctors, nurses, and other health staff to deliver training in evidence based approaches

Collaborators in Cross River State in Nigeria used a strategy that engaged key policy makers and practitioners in producing guidelines (level 4 in the box) in response to a request from the state government to help ensure practice was evidence based for some common conditions. During collaboration with four government and 10 private hospitals, it became clear that there was no experience or understanding of audit or management processes to standardise care. The project, which was recently discussed at Nigeria's National Council on Health (a policy forum) recently, recognised the utility of simple guidelines for increasing the efficiency of the hospital service, and central to this was introducing the basic principles of audit and management processes. Clearly evidence based approaches need people and institutions to have the skills and motivation to evaluate how they do things and how they can improve their practice.

Reflections

If organisations and the individuals within them operated in a machine-like fashion, institutionalising best practice would be straightforward. “Install best practice... if the new part fails to fit smoothly, we just need to give it a good whack to get it in.”¹⁵ Yet health systems are not like this: social interaction and individual beliefs are crucial, as is political support, with particular individuals being in particular places at particular times. As others have highlighted, we need more research into how to identify the components necessary for successful change,¹⁶ while recognising that processes that lead to change in high income settings may not easily be replicated in health facilities in low income countries. The “experiences” we have outlined do not provide evidence of change, but impressions about the process: does this mean that taking the approach outlined in the figure is without a solid evidence base? Trials and systematic reviews try to improve the predictability of medicine, and are applied to research into changing policy and health professional behaviour, so should we conduct trials in changing policy?

Experimental and quasi-experimental designs often do not provide evidence as to what needs to be done to ensure that institutions adopt the principles of evidence based practice; it only becomes efficient to trial an intervention to promote change when the intervention has shown clear potential on a small scale¹⁷ and the pathways for change are delineated. This is where localised audits, case studies, and innovative action research projects can contribute, providing information about the critical factors that determine change. In South Africa the infrastructure is conducive to change, but social interaction seems important to propel it; clinicians in Cross River State, Nigeria, need to learn about audit and how to use it to make their practice more evidence based; and in China high profile opinion leaders are key to ensuring that evidence based obstetric policies become a national priority.

Summary points

Increasing evidence based clinical care in middle and low income countries can have substantive health gains

The World Health Organization uses systematic reviews in forming new guidelines

Changing practice in such countries needs a consistent and persistent approach to ensuring best practice

Change is complex: we should avoid treating policy as if it were a recalcitrant patient

Resource shortages and problems with delivering care are common in middle and low income countries, but they must not be seen as a reason for not attempting to make care more evidence based. With careful thought, many changes, often small and incremental, can be made to improve the quality of care in specific situations, such as reducing the number of routine episiotomies in maternity units, or ensuring that national policies for treating malaria are informed by systematic reviews.

The synthesis of research results only provides one part of the picture, and it is conceited perhaps of people working in systematic reviews to think that their reviews will drop straight into policy. “Clinicians sometimes treat policy as though it is a recalcitrant patient,” a senior policy figure working with an influential UN organisation in a country in Asia stated recently. “If I got a new topic like artemisinin combination treatment in malaria on the national agenda within a year I would be doing well.” So we can answer the question of whether evidence based medicine has changed anything with a qualified yes, but that, at its heart, it is leading to an increasing number of healthcare professionals motivated and equipped to question dogma and authority, to examine evidence scientifically, and to drive appropriate change. These are the people who will ensure countries form national and institutional healthcare policies that are based on reliable evidence.