Rigorous methodology and appropriate process are essential for high quality CPGs development

Clinical practices guidelines (CPGs) represent a fundamental tool in quality of health care improvement, provided that they are developed and implemented through a rigorous process and methodology, [1].

As defined by IOM, CPGs are “statements that include recommendations intended to optimize patient care. These statements are informed by a systematic review of evidence and an assessment of the benefits and costs of alternative care options” [2].

Standards for developing thrustworthy guidelines are established by many international organisations, from the Institute of Medicine (IOM) [3], Guidelines International Network (GIN) [4], the Scottish Intercollegiate Guidelines Network (SIGN) [5], the National Institute for Health and Care Excellence (NICE) [6], the Australian National Health and Medical Research Council (NHMRC) [7], and the World Health Organization (WHO) [8]. Although there is broad consensus on the importance of high quality systematic review in producing CPGs based on the best available evidence, however, most of the published guidelines does not include a systematic, evidence-based methodology to identify, evaluate, summarize and critically interpret findings of all pertinent studies over specific clinical questions [9], [10].

High-quality systematic reviews in the context of CPGs development involve the following steps: definition of scope of the review with aligned specific objectives and research questions; specification of selection criteria and literature search strategy; data extraction; quality appraisal of included studies, summary of findings, and assessment of the certainty of the evidence for each selected critical and important outcome. After the evidence synthesis and interpretation, the panel of experts use the evidence to produce clinical practice recommendations. The higher the overall certainty of the evidence for any critical and important outcome, the more likely a strong recommendation can be made [10].

Surveys on the methodological quality of CPGs [11], indicate that a large number of CPGs are based on moderate to low quality of evidence, as assessed by the Appraisal of Guidelines for Research & Evaluation Instrument (AGREE II) tool [12], the most commonly adopted instrument for assessing the quality of CPG: at international level [12]. In particular, the Domain 3 “Rigour of Development” deals with the process used to collect, critically appraise and summarise the evidence to inform recommendations. The domain consists of the following 6 items, rated on a 7-point scale: (1–strongly disagree to 7–strongly agree: 1. systematic methods used to search for evidence, 2. the criteria for selecting the evidence are clearly described, 3. the strengths and limitations of the body of evidence are clearly described, 4. the methods for formulating the recommendations are clearly described, 5. the health benefits, side effects, and risks have been considered in formulating the recommendations, 6. there is an explicit link between the recommendations and the supporting evidence. The highest score for the item 6 demonstrates that panel of experts used systematically developed evidence synthesis (summary of findings) to inform recommendations [12].

Using non-systematic review methods affects the validity and reliability of the evidence supporting guideline recommendations, producing potentially misleading and untrustworthy results with negative impact on patient outcomes [10].

GRADE Working Group provides one of the most rigorous approach, a framework for assessing the overall certainty evidence, then translating the evidence into recommendations, and judging the strength and the direction of the recommendations.

Recommendations are graded to be weak rather than strong when the certainty of evidence is low, when there is no difference between desirable and undesirable effects, when there is substantial variability or uncertainty in patient values and preferences, and when the applicability of the interventions recommended require substantial resources [13].

ESC published more than 100 guidelines in the European Heart Journal from 1994 to date, covering all the essential topics in cardiovascular medicine. These guidelines supported cardiologist in making decisions about the best interventions to be adopted in the management of patients with cardiovascular diseases [14].

In this issue of the International Journal of Cardiology Heart &Vasculature, Tantawy et al [15] analyzed the ESC clinical practice guidelines in all topics, published on the ESC web site from year 2002 till the last quarter in year 2022, regarding 1. the class of recommendations (COR), either I, IIA, IIB, or III, 2. the level of evidence (LOE) either A, B, or C to detect how consistent is the scientific evidence behind these recommendations.

They identified a total of 4289 recommendations over 37 different topics, with 2140 out of 4289 (median of 49.9%) in Class I and 1825 recommendations (median of 42.6 %) are still in Class II, the latter reflecting an area of uncertainty, because of conflicting evidence and/or a divergence of opinion about the usefulness/efficacy of the given intervention. Regardless the level of evidence, analysis showed the LOE A is only present in 667 (median 15.5%), level B in 1285 recommendations (median 30 %), while, level C was present in 2.337 (median 54.5%) out of the whole recommendations, with wide variability in LOE across different topics. Analysing the topics in the guidelines, the authors found that topics directly related to pharmaceutical medications are richer with clinical trials than other topics that are away from pharmaceutical company interest.

Results provided by this work clearly highlight the importance of systematically evaluate quality of guideline development process, as well the need of prioritising topics with the highest number of recommendations based on expert opinions, small studies and registries, in order to invest in planning and conducting randomized clinical trials.

Beside these conclusions, it can be suggested to assess the quality in CPG development, not only in relation to the distribution of COR and LOE, but considering the six dimensions of AGREE II Instrument, with particular emphasis on the Dimension 3 (Rigour of Development) and Dimension 6 (Editorial Independence). The Dimension 3 refers to the clear description of the different steps of systematic review and the link between evidence and recommendations, specifying the systems adopted for rating the certainty of summarised evidence and for grading recommendations’ strength. The Dimension 6 is concerned with the formulation of recommendations not being unduly biased with competing interests, from funding body and from all members of guideline development group [12].