Reply to Morice et al.: Developing Clinical Guidelines

From the Authors:

We thank Prof. Morice and colleagues for their interest in our study. We are pleased to learn that others share our concerns about the widespread expectation that the Institute of Medicine (IOM) standards for guideline creation will be adopted by guideline developers despite a lack of evidence that adherence to the IOM standards produces recommendations that yield better outcomes than less burdensome approaches.

Our study was designed to explore the importance of the systematic review in shaping guideline recommendations (1). We succeeded in replicating, with a high degree of concordance, those recommendations generated via the IOM standard process. However, the accompanying editorial by Dr. Krishnan argued that despite producing highly concordant recommendations, the Convergence of Opinion on Recommendations and Evidence (CORE) approach is inadequate to replace the IOM approach because the grades were not also highly concordant (2). His rationale was that concordance of the grades is important because grades “influence policies affecting clinical practice (e.g., performance measures and reimbursement by payers).” We would agree that a lack of concordance between the CORE grades and those derived using the Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) approach would be problematic if we had confidence in the validity of the latter. However, although studies demonstrate the reliability (replicability) of grades developed using the GRADE approach with proper training (3), the validity of those grades has been called into question (4).

Should we be concerned about discordance with a flawed comparator? Opinion is likely to be divided. We believe grades derived from the GRADE process and newcomer approaches such as the CORE process should face similar scrutiny and that neither should be awarded a preferred status until their validity is confirmed. We look forward to future explorations of both the GRADE approach and the CORE process to better determine the role of each in the creation of guidelines.