Use of evidence for clinical practice guideline development

Abstract

Clinical practice guidelines provide a framework against which quality of care is measured. Recommendations contained within guidelines are used for decision-making not only within the clinical domain but also other related issues within the health systems. As such the use of research evidence for formulating recommendations contained in a guideline is a global standard to ensure guideline quality. The paper briefly reviews how the need for and use of evidence in guideline development shaped up historically and then provides an overview of the four typologies of guideline development mechanisms at the country level.

INTRODUCTION

Clinical practice guidelines (guidelines) have been a familiar part of clinical practice for decades, but their importance and consequent proliferation worldwide is a more recent phenomenon. The increasing interest in guidelines is due to several health system factors – the need to cut costs, newer health care financing arrangements, variation in health service delivery mechanisms, and the transition of medicine toward more multidisciplinarity. Guidelines are often included in legal contracts between providers and purchasers due to the need to contain rising health-care costs[1,2] and concerns about overuse of interventions and investigations.[3] Guidelines thus are being used not only for clinical decision-making but also as a standard framework for measuring quality of care. Guidelines thus provide the technical basis for setting rules of procedures for operation of health facilities and clinics, define health-care budgets, and spending, determination of what costs should be reimbursed by insurers, formulating essential medicine lists, and influence pharmaceuticals-related trade policies.

The increasing importance of guidelines has meant that the recommendations within it, and consequently, the methods used to formulate these recommendations has come under increased scrutiny. In response, the concept of guidelines and the way it is being developed has also evolved over time. The review discusses this evolution with a focus on the use of evidence in the guideline development process and also provides an international overview of guideline development process at the country level.

THE NEED FOR EVIDENCE IN DEVELOPING GUIDELINES

The modern history of guidelines began in 1992, with the Institute of Medicine, United States of America (IOM, USA) published a report titled “Clinical Practice Guidelines: Directions for a New Program” which for the first time formally defined guidelines.[4] This marked the beginning of an era, where the profile of guidelines increased exponentially owing to several health system issues. During this period, the number of guidelines increases rapidly from just 374 indexed in MEDLINE in 1993 to 980 in 1996.[5] Even though the IOM, USA, document had noted the importance of having a systematic process of development of clinical practice guidelines,[4] it continued to be based on expert opinions. Research indicated that the ability of a guideline to improve quality of care and improve health system performance depended on several factors during the guideline development process. Expert opinions varied not only between different specialties of expert clinicians but also from within clinicians of the same specialty and between those clinicians who made guidelines and clinicians who actually implemented them.[6,7,8] It was also found that the composition of the guideline development group influenced the recommendations[9,10] and that the recommendations made by clinical experts were often in conflict with the best available research evidence.[11] A rigorous process of identifying and evaluating research evidence and adapting them appropriately in a transparent manner with effective management of conflict of interests during the guideline development process[12,13] was thus found to be critical for a guideline to be able to provide a meaningful framework against which medical quality of care can be measured and important health systems' decisions be based on.

USE OF EVIDENCE FOR GUIDELINE DEVELOPMENT

The use of research evidence for developing guidelines has its roots in of two larger movements which took the world of medicine and health care by storm since the 1990s.

The first of these movements was a result of the HIV/AIDS epidemic with the USA as it is the epicenter. HIV/AIDS activists in the USA constantly argued that clinical trials were not only sites for scientific activity but also sites for clinical care. However, an insurance-based healthcare financing system in the USA meant coverage could be denied based on clinical practice guidelines. For activists, this implied that the focus of their activity could no longer be restricted to demands for increased funding for research. The activists demanded greater say in understanding the use and interpretation of research evidence to formulate recommendations for guidelines. Activists not only used their political power and took on professional societies, insurance companies to secure a place in drug development, regulation and guidelines development panels but also helped forge “new social relationships and identities, new institutions, and new facts and beliefs in the process.”[14] Activists were not only vocal about how their priorities were different from those of clinicians resulting in negative clinical outcomes but were also able to project themselves as credible participants in design, conduct, and interpretation of clinical trials for formulating recommendations – an arena that had always been exclusive for credentialed experts.[15]

Patient or lay participation in health care decision-making had implications beyond AIDS as it broke the cultural authority health-care professionals till then have been able to hold on to for centuries. It changed the dynamic of the doctor-patient relationship and blurred the boundaries between society and science.[14] This was a subtle but firm change in the guideline ecostructure as opinions of experts were no longer taken for granted, even in matters which were technical in nature. It has changed the very nature of what constitutes as being “credible” in health policy or guideline development process.

The evidence-based medicine movement on both sides of the Atlantic was the other movement which had a more profound and direct influence in advocating for the greater role of evidence in guideline development and this continues till date. Although the roots of evidence-based medicine have been planted much[16] earlier the movement caught steam only in 1992 when researchers from Canada and USA collaborated with each other to form the evidence-based medicine working group.[17] Across the Atlantic in United Kingdom, the Cochrane Collaboration was formed in 1993.[16] The practice of evidence-based medicine was described as “ integrating individual clinical expertise with the best available external clinical evidence from systematic research.”[18]

The requirement of guideline panels to be objectively informed about the current status of research evidence on efficacy and safety of alternative health-care interventions is well fulfilled by a systematic review. A systematic review systematically searches for all available research evidence (published or unpublished), critically evaluates the quality of available evidence, and provides an evidence summary for a particular intervention in comparison to an alternative. The element of transparency of judgments used in critical appraisal of quality of evidence in a systematic review additionally allows guideline panels to make judgments about their utility and acceptability. In addition, systematic reviews by collating all evidence on a particular research question into a single study serves a practical purpose in the current age of “information overload”[19] where it is impossible for any single health professional to be updated of all evidence. It also prevents any bias in selection of individual studies for informing guideline development. Guideline panels use systematic reviews of health-care interventions, diagnostic tests, prognostic markers, cost-effectiveness analyses, and even systematic reviews of qualitative studies to understand values, perceptions, and preferences of patients, health-care staff, and policy makers to inform the guideline development process.

The influence of these two movements in changing the concept of guidelines has been reflected in the definition of guidelines given by major international organizations over time and has been summarized in Table 1. Over time organizations had included the need for research evidence and later specifically inculcated the need for systematic reviews in informing the guideline development process in an objective and transparent manner. As, for example, in 2007, it was found that recommendations in the World Health Organization (WHO) guidelines were not based on systematic search and appraisal of evidence but on expert opinions.[26] In response to the ensuing public outcry, the WHO responded by developing a Guideline Review Committee to implement and oversee quality assurance mechanisms for guideline development. A follow-up evaluation in 2013 found that the quality of WHO guidelines have improved[27] on several counts.

Table 1.

Evolution of definitions of guidelines over time in different organizations

The use of evidence in the form of systematic reviews is now considered as a standard internationally for guideline development. However, systematic reviews do not provide any information on how much confidence can be placed on a recommendation made on the basis of the evidence from the systematic review and how applicable it might be in a particular setting and how well it aligns to health systems values and preferences. To overcome these issues, a system called Grading of Recommendations: Assessment, Development, and Evaluation (GRADE) which enables a guideline panel to classify a recommendation as strong or weak based on a series of sequential, explicit judgments to balance harms and benefits between alternative strategies, quality of evidence, resource, and cost implications as well as variability in values and preferences[28] has been developed. The GRADE approach has been adopted by more than hundred organizations globally, including the WHO[25] and is now considered the standard for this purpose replacing the previous methods of recommendations being based on quality or levels of evidence only.

A key issue during guideline development, during formulation of recommendations, is acquiring consensus among a diverse group of individuals, often with competing interests and values. While moderated group discussions are commonly used, such an approach does not encourage equal inputs from all participants. The use of formal procedures such as the Delphi technique or the nominal group technique are hence essential for ensuring guideline quality.[24,25]

ASSESSMENT OF QUALITY OF GUIDELINES

While there are several frameworks available to understand the quality of guidelines, a conceptual framework to understand how guidelines impact health outcomes measured is presented in Figure 1. The framework, based on a 2001 report,[29] indicates that several key strategies during guideline development such as involvement of health system actors, consideration of context, and methodological rigor are essential for successful implementation of guidelines and consequential improvement in health outcomes.[1,30,31] Quality of guidelines is thus appraised by evaluating strategies used in various parts of the guideline development process. A review of guidelines from 10 European countries and Canada[32] and 18 major guideline development program[33] by the Appraisal of Guidelines, Research, and Evaluation (AGREE) collaboration led to the identification of factors in the guideline development process which predicted the quality of guidelines. Further research by the AGREE collaboration led to the development of an instrument to measure the quality of guidelines. It was called the AGREE tool and had 23 items to be graded on a 4-point Likert scale.[34] The tool was well accepted internationally and was subsequently revised to improve usability, reliability, and validity for a revised version called AGREE II. The AGREE II tool has been tested and validated rigorously[35] and is used by several leading organizations including the WHO[25] to monitor the quality of guidelines.

Figure 1.

Conceptual framework to understand how guidelines improve health outcomes[29]

GUIDELINE DEVELOPMENT AT THE COUNTRY-LEVEL

This section of the review provides an international overview of how guidelines are developed at the country level. It does not intent to exhaustively cover guideline development for all the countries of the world but attempts to present the broad typologies of how guidelines are being developed at the country level.

Countries with a central agency responsible for guideline development

Guideline development in these countries is the exclusive mandate of a central agency. Countries which have adopted this model are all high-income countries with a strong public health system providing universal health coverage. This ensures easier adoption and implementation of a single guideline throughout the country. However, the statutory footing and level of autonomy these centralized agencies enjoy are different in different countries.

National Institute for Health and Care Excellence (NICE) in England is funded fully by the Department of Health but is operationally independent of the government.[36] The organization has a very detailed manual on guideline development which details the process for guideline panel formation, stakeholder involvement, development of evidence summaries, and formulation of recommendations.[24]

Scottish Intercollegiate Guidelines Network (SIGN), Scotland, is a part of Healthcare Improvement Scotland, a government organization since 2005. The SIGN guidelines are prepared on the basis of three principles – multidisciplinary, nationally representative panels, identification and critical appraisal of evidence through systematic reviews, and recommendations being exclusively linked with supporting evidence.[37] To prevent duplication of efforts SIGN also adapts already existing NICE guidelines[38] using a tool for contextualizing guidelines called ADAPTE tool.[39]

The Finnish Medical Society Duodecim, which is entrusted with the responsibility of developing guidelines, is not a public agency. It is the country's largest scientific association. The guidelines, known as “current care guidelines,” are prepared by the Duodecim in a collaborative manner with various other medical specialist organizations but are funded publicly.[40] In Finland, guidelines are integrated with the evidence-based medicine electronic decision support system, which allows health-care professionals to open guidelines from within the electronic patient record thus enabling greater dissemination and uptake.

Quality of guidelines in these countries is generally good and has improved over the years on account of provision of methodological training for guideline development as well as use of the AGREE II tool as a benchmark for guideline development.[24,37] In England and Scotland (both under the United Kingdom), financial incentives are often used to facilitate guideline implementation.[41,42]

Countries with a centrally coordinated mechanism with multiple organizations involved in the guideline development process

To improve quality of guidelines, some countries have adopted a different model wherein they have a centralized agency with the mandate to coordinate and endorse guidelines instead of actually taking up the guideline development function. Countries which have adopted this model are high income countries and most of them have a pluralistic health system. Any health system actor, be it a professional society or a public agency, can develop guidelines as per a preset standard and this has to be endorsed before implementation by the centralized agency. Prominent countries which use such a model are Germany, France, Australia, USA, and Canada. Governance, autonomy as well as regulatory frameworks are widely different in different countries.

In Germany, guidelines are developed by individual medical associations, but the process is coordinated and endorsed by the Association of the Scientific Medical Societies in Germany (AWMF – Arbeitsgemeinschaft der Wissenschaftlichen Medizinischen Fachgesellschaften), which is an umbrella organization of all medical associations. AWMF is not publicly funded. AWMF also coordinates with the German Medical Association (Bundesärztekammer) and the National Association of Statutory Health Insurance Physicians (Kassenärztliche Bundesvereinigung) to develop guidelines for disease covered under the publicly funded program called “Nationale Versorgungsleitlinien.”[43] These guidelines are mandatory in nature and can be used as the basis for malpractice litigations.

In France, the Haute Authorité de Sauté develops as well as approves guidelines developed by external agencies.[44] similar to what is being done in Germany.

Australia is slightly different from other countries because the National Health and Medical Research Council (NHRMC) performs a dual function of developing guidelines for the government as well as approving guideline developed by others. NHRMC is a public agency and has a strong legislative footing through the NHRMC Act 1992.[45] Medical societies as well as provincial authorities and other agencies are free to develop their own guidelines as per the standards laid down by the NHRMC which scrutinizes and approves them before dissemination and implementation.[46]

In USA, guidelines are developed by several agencies and are submitted to the National Guideline Clearing House, an arm of the Agency for Healthcare Research and Quality, for assessment as per its standard criteria.[47] The U.S. Preventive Services Task Force is a well-recognized independent group which develops evidence-based guidelines for preventive services such as screening tests, counseling, or preventive medications.[48] Canada too follows a model similar to USA in terms of guideline development.

In most countries which fall under this bracket, quality of guidelines is good and either the AGREE II or its variants is used for quality control and the coordinating agency is usually involved in capacity building efforts.

Countries where multiple organizations develop guidelines with no mechanism for coordination

Most low- and middle-income countries, including India, and few high income countries (such as Korea and Singapore) fall under this bracket where a multitude of actors develop their own guidelines with no centralized agency to either coordinate or develop guidelines. An evaluation of guidelines in South African countries had found that the quality of guidelines was poor and it varied both within and between countries.[49] Nothing more is known about the quality of guidelines or about the use of evidence in the guideline development process. There is a little commonality in the manner in which health systems are arranged in these diverse groups of nations either.

Countries which do not develop their own guidelines

This group consists of several low-income countries, including some with political instability, which do not develop their own guidelines. Such countries are usually dependent on foreign aid and hence follow guidelines either developed by the WHO or any such as enforced by the donor. Almost nothing is known about quality of guidelines and guideline development in these countries or the suitability of the guidelines being used for the country. Although the ADAPTE tool[39] is very suitable for use in such a context, its use at the country level is minimal if at all and this has not been documented in the medical literature.

CONCLUSION

Guideline development has undergone changes in the last few years with the use of research evidence, in the form of systematic reviews in formulating recommendations becoming an acceptable global standard. Historically, several forces and health system issues have triggered this change. However, for most countries globally, where there is no formal organization or coordination on guideline development, and very little is known about processes being used for guideline development. The quality of guidelines used in these countries, how evidence is used in formulating them and the barriers to uptake of evidence are critical research gaps which need to be studied in the future. Such research can help facilitate understanding of mechanisms that can be setup within different health systems for optimal development of good quality guidelines which can lead to improved health outcomes.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.