Clinical Practice Guideline Methodology: An Evolving Science

Clinical practice guidelines aid in making medical decisions, with the goal of improving healthcare quality and outcomes. The methodology used to develop clinical practice guidelines has evolved over time, with codification of an approach in 2011 when the Institute of Medicine (now the National Academy of Medicine) published standards for guideline creation (1). The standards have been lauded by the guideline development community for their promotion of rigorous, unbiased, transparent clinical recommendations. Previously, guideline recommendations were based on informal literature reviews, clinical experience, and opinion, with consensus derived by discussion.

A key feature of the standards for guideline creation is the requirement that clinical recommendations be informed by systematic reviews of the evidence (1). Systematic reviews can be resource intensive and time consuming to perform. As a result, some guideline development organizations began looking for equally rigorous alternatives to doing a full systematic review for each clinical recommendation.

Within the American Thoracic Society, a unique opportunity presented itself when eight completed but unreleased guidelines were awaiting publication. Investigators composed a panel of experts for each guideline topic, asked them the same questions that were posed in the guidelines, and used a consensus-building process called Convergence of Opinion on Recommendations and Evidence (CORE) to formulate clinical recommendations (2). The CORE-derived recommendations were compared with the completed guideline recommendations that were informed by systematic reviews of the evidence in compliance with the Institute of Medicine’s standards. The investigators found 98% concordance in the recommendations when there was greater than 70% agreement among the experts. These results were subsequently validated in studies that looked at clinical recommendations related to idiopathic pulmonary fibrosis (3) and community-acquired pneumonia (4). In sum, accumulating evidence suggests that not all guideline questions require a full systematic review of the evidence to formulate a clinical recommendation.

In this edition of the Journal (pp. 17–28), clinical practice guidelines addressing pediatric liberation from mechanical ventilation are provided by the PALISI (Pediatric Acute Lung Injury and Sepsis Investigation) Network (5). These are the first guidelines to incorporate the CORE process into their creation. The guideline committee asked 11 questions. Three yielded recommendations using the CORE process. The remaining eight questions were informed by five systematic reviews of the evidence. The net effect was that five systematic reviews were required rather than eight, saving resources, time, and effort. Controversy concerning the CORE process is based largely on the slippery-slope fallacy; that is, there are concerns that the CORE process may be applied so broadly that systematic reviews may become extinct (6–11). The PALISI Network guideline is an excellent example of how the CORE process can be incorporated into guideline development in a prudent fashion.

The CORE process is not the only example of methodological ingenuity that exists within the PALISI Network guideline, however. Clinical trials most often compare an intervention against no intervention. The comparison may be phrased as intervention A versus no intervention, or intervention A plus standard care versus standard care alone. Regardless of the wording, lack of comparison with an alternative intervention lends itself to series of guideline recommendations such as “we recommend intervention A,” “we recommend intervention B,” and “we recommend intervention C,” leaving readers wondering which of the recommended interventions should be implemented first, second, and so on.

Network meta-analysis is a relatively new approach to making indirect comparisons in the context of a systematic review. If there are trials that compare intervention A with no intervention and trials that compare intervention B with no intervention, network meta-analysis enables comparison of intervention A and intervention B (Figure 1). When incorporated into a guideline, network meta-analysis provides another level of guidance to readers, prioritizing various interventions the committee has concluded are beneficial.

Figure 1.

Network meta-analysis diagram. The size of each node indicates the number of subjects who received the intervention. Lines indicate that studies directly compared the interventions, with the thickness of the line indicating the number of such studies. All interventions are indirectly compared in network meta-analysis.

The PALISI Network guidelines are an example of the usefulness of network meta-analysis in guideline development. The authors collected trials that compared early low-dose corticosteroids versus no corticosteroids, early high-dose corticosteroids versus no corticosteroids, late low-dose corticosteroids versus no corticosteroids, and late high-dose corticosteroids versus no corticosteroids and used network meta-analysis to compare the four interventions (12). As a result, the guideline committee was able to conclude that early corticosteroids were most important to avoiding upper airway obstruction among patients at high risk, with early high doses and early low doses performing similarly. Without a network meta-analysis, the guideline committee may have had difficulty or may not have been able to reach similar conclusions. The PALISI Network guidelines further support a growing trend toward network meta-analysis in guideline development.