Temporal trends in evidence supporting therapeutic interventions in heart failure and other European Society of Cardiology guidelines

Abstract

Aims

This study aimed to determine whether any change occurred over time in level of evidence (LoE) of therapeutic interventions supporting heart failure (HF) and other European Society of Cardiology guideline recommendations.

Methods and results

We selected topics with at least three documents released between 2008 and April 2022. Classes of recommendations (CoR) and supporting LoE related to therapeutic interventions within each document were collected and compared over time. A total of 1822 recommendations from 18 documents on 6 topics [median number per document = 112, 867 (48%) CoR I] were included in the analysis. There was a trend towards a reduction over time in the percentage of CoR I in HF (46–36–34%), non‐ST elevation myocardial infarction (NSTEMI; 78–58–54%), and pulmonary embolism (PE; 65–50–39%) guidelines, with a decrease in the total number of recommendations for HF only. Percentage of CoR I was stable over time around 40% for valvular heart disease (VHD) and atrial fibrillation (AF), and around 60% for cardiovascular prevention (CVP), with an increase in the total number of recommendations for VHD and CVP and a decrease for AF. Among CoR I, 319 (37%) were supported by LoE A, with a decrease over time for HF (56–46–42%), an increase for NSTEMI (29–38–48%) and AF (28–31–36%), a bimodal distribution for PE and CVP, and a lack for VHD.

Conclusions

LoE supporting therapeutic recommendations in contemporary European guidelines is generally low. Physicians should be aware of these limitations, and scientific societies promote a greater understanding of their significance and drive future research directions.

Introduction

Guidelines are the key instrument for the implementation of evidence‐based medicine worldwide, and adherence to guideline recommendations is associated with an improvement of outcomes in patients with cardiovascular (CV) diseases. ¹ , ² Recommendations reported in guideline documents are based on the highest level of evidence (LoE) available, which over the last 40 years has been obtained mainly through randomized clinical trials (RCTs). RCTs have transformed medical practice and have become the gold standard for assessing the effectiveness and safety of interventions. ³ An LoE is assigned to each recommendation in guidelines and the highest LoE is given when multiple RCTs (sometimes combined into meta‐analyses) support or oppose a certain diagnostic or therapeutic procedure. ⁴ Nonetheless, two major review manuscripts published in 2009 and 2019 reported that <15% of recommendations from over 100 international guideline documents published in the last four decades were supported by evidence from RCTs, without any important change over time. ⁵ , ⁶ Notably, these review manuscripts were primarily focused on American rather than European guidelines, included guidelines from all CV conditions without a comprehensive analysis for each topic, and had last document abstracted in 2018. ⁵ , ⁶

Heart failure (HF) is one of the leading CV health problems in Europe and worldwide, and first guidelines for this condition were issued by the European Society of Cardiology (ESC) in 1995. ⁷ Since then, the ESC HF guideline document underwent several updates, until the most recent one published in late 2021. ⁴ The overall LoE supporting current ESC HF guideline recommendations is unclear, as well as whether any significant change in LoE occurred in ESC HF guidelines over recent years. How evidence of ESC guidelines for HF compares with that supporting contemporary ESC guidelines for other CV conditions is also undetermined. Thus, the aim of this work was to investigate temporal trends of LoE behind therapeutic interventions included in the ESC HF guidelines of the last decade and compare these trends with those of contemporary ESC guidelines for other acute and chronic CV conditions. Reasons for these trends and areas of potential improvement were then discussed.

Methods

Guideline documents selection

We identified ESC guidelines posted on the ESC website as of April 2022. Only comprehensive guideline documents with recommendations organized by classes of recommendations (CoR) and LoE were included. Expert consensus documents and focused updates were excluded. To allow for a more unbiased temporal trend evaluation, we selected those topics that had at least three comprehensive guideline documents published within the last 15 years. This time span was selected considering that a systematic presentation of CoR and LoE was not available in guidelines published before the late 2000s. In addition, the policy of the ESC mandates that guidelines are updated each 5 years approximately.

Definition of classes of recommendations and level of evidence

As stated in most recent ESC guideline documents, CoR I are those for which there is ‘Evidence and/or general agreement that a given treatment or procedure is beneficial, useful, effective’ and were the focus of our present work. These recommendations may be supported by different LoEs, classified as follows: LoE A: ‘Data derived from multiple randomized clinical trials or meta‐analyses’; LoE B: ‘Data derived from a single randomized clinical trial or large non‐randomized studies’; and LoE C: ‘Consensus of opinion of the experts and/or small studies, retrospective studies, registries’.

Search methodology

Guideline documents were downloaded (https://www.escardio.org/Guidelines/Clinical‐Practice‐Guidelines) and abstracted by two independent reviewers (M. C. and V. D. M.). Number of recommendations, together with their CoR and LoE, was collected from tables included in these documents. We elected to investigate only recommendations related to therapeutic interventions according to the goals of our analysis and because of the wide heterogeneity of diagnostic interventions in different guideline documents. Therapeutic interventions included both medications and procedures listed within each table. Rows in each table were counted and grouped according to their CoR and LoE. When multiple interventions were reported within a single row, a unique CoR and LoE was assigned, although details contained in each row were separately listed and reviewed. For example, at page 2167 from HF guidelines 2016, ⁹ in the table entitled ‘Recommendations for the treatment of hypertension in patients with symptomatic heart failure with reduced ejection fraction’, it reads the following: ‘ACE inhibitor (or ARB), a beta‐blocker or an MRA (or a combination) is recommended to reduce blood pressure as first‐, second‐ and third‐line therapy, respectively, because of their associated benefits in HFrEF’, with CoR I and LoE A. Multiple interventions are listed, but a unique CoR and LoE is assigned.

Statistical analysis

We reported frequencies and percentages of therapeutic CoR I for each guideline document, as well as the proportion of CoR I classified as LoEs A, B, and C. Absolute number of recommendations and CoR I changed over time and by topic; thus, these absolute numbers were also reported in detail.

Results

Guidelines from the following topics having at least three subsequent full documents published over time were included: HF (2012, ⁸ 2016, ⁹ 2021 ⁴ ), non‐ST elevation myocardial infarction (NSTEMI; 2011, ¹⁰ 2015, ¹¹ 2020 ¹² ), pulmonary embolism (PE; 2008, ¹³ 2014, ¹⁴ 2019 ¹⁵ ), valvular heart disease (VHD; 2012, ¹⁶ 2017, ¹⁷ 2022 ¹⁸ ), atrial fibrillation (AF; 2010, ¹⁹ 2016, ²⁰ 2020 ²¹ ), and CV prevention (CVP; 2012, ²² 2016, ²³ 2021 ²⁴ ).

Overall, 1822 therapeutic recommendations from 18 guideline documents published between 2008 and 2022 were included in the analysis. The median number of recommendations per document was 112 (min 26 PE in 2008, max 193 AF in 2010), and 867 were CoR I (48%), 549 CoR IIa (30%), 266 CoR IIb (14%), and 140 CoR III (8%).

There was a trend towards a reduction in the percentage of CoR I in guideline documents analysed for HF, NSTEMI, and PE, although the total number of recommendations in each document decreased over time for HF but increased over time for NSTEMI and PE (Figure  1 ), making the decrease with time in CoR I observed in HF guidelines a matter of even greater interest. The percentage of CoR I was stable over time for VHD and CVP, and steadily around 40% for VHD and 60% for CVP; both topics showed an increase in the total number of recommendations in each document over time (Figure  1 ). Percentage of CoR I was around 40% in AF guideline documents, although with a drop in the 2016 document (Figure  1 ). Earlier AF and HF guidelines started off with the highest total number of recommendations (i.e. >100) among all guideline documents published in the years 2008–2012, whereas contemporary HF guidelines included <100 recommendations.

Figure 1.

Prevalence of class I recommendations over time. AF, atrial fibrillation; CVP, cardiovascular prevention; HF, heart failure; n, number of class I recommendations over total recommendations in each guideline document; NSTEMI, non‐ST elevation myocardial infarction; PE, pulmonary embolism; VHD, valvular heart disease.

Among class I recommendations from all 18 documents, 319 (37%) were supported by LoE A, 280 (32%) by LoE B, and 268 (31%) by LoE C. When comparing different topics, average percentage of class I recommendations with LoE A was higher for CVP guidelines (59%), followed by HF (48%), NSTEMI (39%), AF (32%), and PE (27%). The temporal trend in the percentage of LoE A class I recommendations decreased over time in HF, increased in NSTEMI and AF, and had a bimodal distribution in PE and CVP (Figure  2 ). There was a lack of LoE A for VHD class I recommendations (Figure  2 ).

Figure 2.

Distribution of level of evidence among class I recommendations over time. AF, atrial fibrillation; CVP, cardiovascular prevention; HF, heart failure; n, number of level of evidence A recommendations over class I recommendations in each guideline document; NSTEMI, non‐ST elevation myocardial infarction; PE, pulmonary embolism; VHD, valvular heart disease. The total number of class I recommendations for each guideline document is listed in Figure 1 .

Discussion

Temporal trend of evidence in European Society of Cardiology guideline documents

In this analysis of the trend of evidence supporting ESC guideline documents, we found a reduction over time in the percentage of class I recommendations in HF, NSTEMI, and PE guidelines, although HF guidelines were the only ones also showing with time a reduction in both total number of recommendations and class I recommendations supported by the highest LoE A. Guideline documents for NSTEMI and AF showed an opposite trend, and those for CVP showed the highest average percentage of LoE A class I recommendations among all. This may be due in part to major therapeutic advancements made in areas other than HF, including new antiplatelet agents in NSTEMI (discussed by Szummer et al. ²⁵ ), widening indications for oral anticoagulants and transcatheter ablation in AF (summarized by Kirchhof et al.), and a more extensive use of CV medications in primary and secondary CVP (discussed by Sharma et al. in relation to glucose‐lowering therapies, ²⁶ by Preiss et al. in relation to lipid‐lowering therapies, ²⁷ and Pfeffer and McMurray in relation to hypertension therapies ²⁸ ). The vast successes achieved in the treatment of HF with reduced ejection fraction (HFrEF) from the early 1990s until the late 2010s (nicely summarized by Tomasoni et al. ²⁹ ) likely explain the blooming of recommendations in earlier HF guidelines, with a subsequent drop in 2021. Latest HF guidelines increasingly focused on left ventricular ejection fraction categories, and the advent of HF with preserved ejection fraction (HFpEF) introduced a high degree of uncertainty in HF guidelines, primarily due to neutral results of RCTs testing HFrEF medications in HFpEF. This started with the publication of the CHARM‐Preserved in the early 2000s, ³⁰ followed by the I‐PRESERVE ³¹ and TOPCAT, ³² until the most recent PARAGON‐HF, ³³ and likely contributed to the reduction of class I LoE A recommendations observed over time. Notably, successful experiences with sodium‐glucose cotransporter 2 inhibitors in HFpEF ³⁴ , ³⁵ were not incorporated in the latest HF 2021 ESC guideline document. In addition, some recent ‘negative trials’ in HF other than HFpEF may have played a role. For example, comparing 2016 vs. 2021 HF guideline documents, recommendation for primary prevention implantable cardioverter defibrillator (ICD) in non‐ischaemic HFrEF went from class I to class IIa following the DANISH trial, ³⁶ as well as cardiac resynchronization therapy in HFrEF with a left‐bundle branch block duration of 130–149 ms following an individual patient meta‐analysis of five randomized trials, ³⁷ and myocardial revascularization in ischaemic HFrEF with angina and optimized medical therapy following the STICH trial. ³⁸

The past and the future of clinical trials in cardiovascular diseases and heart failure

HF remains an area of intense clinical research among CV diseases, and LoEs are expected to be the highest in areas that have been the most frequent focus of RCTs. A recent report investigating clinical trials with therapeutic interventions (both drugs and devices) for CV diseases recorded in the Clinicaltrials.gov database between January 2013 and December 2018 found that the most studied primary fields were arterial hypertension (23%), coronary artery disease (20%), and HF (10%), followed by cardiac arrhythmias (9%), stroke (7%), vascular diseases (5%), and dyslipidaemia (5%). ³⁹ Interestingly, in our analysis, latest CVP guidelines were those that had the highest percentage of class I LoE A recommendations among the most recent ones, and NSTEMI and AF guidelines showed a steadily increasing trend in LoE A over time (Figure  2 ). On the contrary, our data suggest that clinical research in HF therapeutics has not resulted into a similar proportionate increase in the LoE supporting HF guidelines but rather into a decline. Notably, three quarters of CV studies recorded in the Clinicaltrials.gov database analysis were randomized ones, and randomization was independently associated with greater likelihood of results publication. ³⁹

Both ESC and American College of Cardiology/American Heart Association (ACC/AHA) guidelines assign an LoE A when supporting data are derived from multiple RCTs or meta‐analyses, but the revised ACC/AHA methodology also allows for LoE A status in the presence of ‘one or more RCTs corroborated by high‐quality registry studies’. ⁴⁰ Although observational studies lack randomization of interventions, high‐quality registries have the potential to confirm results from RCTs and to extend them to individuals with more complex comorbid conditions. ⁴¹ A new advancement in generating evidence‐based knowledge is offered by registry‐based randomized controlled trials (RRCTs), which use registries as a platform for case records, data collection, randomization, and follow‐up. Participants of RRCTs are enrolled in real‐world clinical practice and randomly assigned into an experimental group or a control group to test the effectiveness of a specific intervention. The impact of treatment is usually assessed on a range of outcomes over a prolonged follow‐up period, linking data from high‐quality registries to electronic health records (EHRs). ⁴² The availability of an operative EHR system is key to performing RRCT, and this still represents a primary limitation in several countries. ⁴³ There have been several RRCTs in areas other than HF, in which the first RRCT entitled ‘Spironolactone Initiation Registry Randomized Interventional Trial in Heart Failure with Preserved Ejection Fraction (SPIRRIT‐HFpEF)’ should soon be completed in Sweden. This study will evaluate the impact of spironolactone, a drug generically approved for HF, in real‐world HFpEF patients, with the goal of validating the conflicting results of the original RCT TOPCAT. ³² Pragmatic clinical trials (PCTs) with broader eligibility criteria, shorter duration, and remote‐only follow‐up have also been proposed, with the goal of overcoming traditional RCT challenges and producing more generalizable results. An example is represented by the ‘ToRsemide compArisoN With furoSemide FOR Management of Heart Failure’ (TRANSFORM‐HF) trial, ⁴⁴ designed with the aims of comparing the effects of torsemide vs. furosemide among patients with HF in the United States. Insufficient evidence exists to conclude that torsemide should be routinely recommended over furosemide, ⁴⁵ although the current dominant use of furosemide is related more to its long‐time clinical experience rather than its evidence. In this ongoing PCT, patients are enrolled and randomized at any time during an HF hospitalization; dosing and frequency changes to the randomized therapy after hospital discharge are at the discretion of the patient's usual outpatient clinicians; no study‐specific, in‐person follow‐up visits are planned but periodical phone interviews from a centralized call centre for data collection. Many experts believe that this ‘rough’ methodology intentionally chosen to ‘represent the real world’ may lead to findings that could simply represent the play of chance rather than true associations, and an intense debate over ‘minimum standards’ of future clinical trials generating real‐world evidence is ongoing. ⁴³ , ⁴⁶

Comparing trends in European and American guideline documents

Although the ESC and ACC/AHA guidelines use similar evidence to generate recommendations, a recent analysis found that a greater proportion of recommendations in the ESC guidelines were classified as LoE A. In particular, Fanaroff and colleagues investigated LoEs supporting 26 ACC/AHA and 25 ESC guidelines published between 2008 and 2018 and found that <10% of recommendations from ACC/AHA guidelines and <15% of recommendations from ESC guidelines (both diagnostic and therapeutic ones) were supported by evidence from multiple high‐quality RCTs and characterized as LoE A. ⁵ Authors also compared these most recent guideline documents with their immediate predecessors published in the years 1999–2004, and they demonstrated that the proportion of LoE A recommendations did not increase in either ACC/AHA or ESC guidelines ⁵ and that results were remarkably similar to those published in 2009 by Tricoci and colleagues. ⁶ Among class I recommendations only, they found that ~15% were supported by LoE A in ACC/AHA and ~22% in ESC documents, with higher percentages of LoE A for general cardiology and coronary artery disease, average ones for HF and myocardial disease, and lower ones for vascular medicine, VHD, and electrophysiology. ⁵ We found that the prevalence of LoE A supporting class I therapeutic recommendations of most recent ESC guidelines was on average 35%, with a peak of 61% for CVP 2021, followed by 48% for NSTEMI 2020, 42% for HF 2021, 36% for AF 2020, 24% for PE 2019, and 6% for VHD 2021, with a steady decline over time for HF only (Figure  2 ). Thus, there seems to be an increase compared with the period before 2009 in the LoE supporting most contemporary ESC guidelines. No updated analysis including most recent ACC/AHA guidelines has been released, but differences are likely to persist. ⁵ As for HF, a head‐to‐head comparison of most recent ESC vs. ACC/AHA guidelines found that several topics showed conflicting recommendations, even when reviewing the same published evidence. This is the case for example of some therapeutic recommendations, including sacubitril/valsartan in HFrEF (I A in ACC/AHA vs. I B in ESC) or ICD implantation in non‐ischaemic HF (I A in ACC/AHA vs. IIa A in ESC). ⁴⁷ Similar comparisons between ESC and ACC/AHA guidelines have recently been made for other conditions such as hypertension, ⁴⁸ with the intent of harmonizing future documents and catalysing changes in practice that would lead to improved prevention, awareness, treatment, and control of CV diseases.

Role of guideline documents beyond evidence‐based practice

Finally, ESC guidelines ‘aim to present all the relevant evidence on a particular clinical issue in order to help physicians weigh the benefits and risks of a particular diagnostic or therapeutic procedure’ and ‘provide the best possible advice to practicing physicians, clarify contemporary areas of consensus and disagreement, improve standards in clinical practice, and help everyday clinical decision‐making’. Publication of guidelines has undoubtedly determined a standardization and improvement of care over time, at the advantage of all citizens worldwide. Although CV diseases remain the leading cause of mortality in Europe, CV disease mortality is now decreasing in nearly all European countries including those of Central and Eastern Europe, as recently reported in the 2019 ESC Atlas and subsequent updates. ⁴⁹ Implementation of guidelines and consequent increasing of the standards of CV disease care are certainly at the origin of this success, although a direct causal role is difficult to establish. Nonetheless, our work demonstrates that the evidence on which guideline documents are based is still largely lacking, and all clinicians should be aware of this important limitation when referencing to these documents, particularly when facing prosecution/litigation. An in‐depth discussion around the legal role of guidelines goes beyond the scope of our work. Guideline documents are being used in the malpractice arena to define a credible standard of care to measure, in different ways across countries, the accused physician for an alleged problem addressed, despite a medical society's disclaimer that they are not intended, nor devised, for that purpose. ⁵⁰ In light of the significant gaps in evidence highlighted in our present work and in previous ones, ⁵ , ⁶ authors of guidelines should increasingly consider the potential future courtroom use of these documents as they attempt to use evolving knowledge to enhance patient care.

Limitations

This analysis has several limitations. We elected to analyse only therapeutic recommendations from tables included in each guideline document to increase the reproducibility and consistency of our findings. Thus, further recommendations included in the text of each guideline documents may potentially have been missed. Similarly, addition of number of references and of specific RCTs, meta‐analyses, and observational studies cited in each guideline document would have increased the accuracy of our work. However, these citations are scattered in the text, and references are cited differently in tables and text within each guideline documents. We neither considered guideline updates nor consensus statements and documents, for reasons that we discussed and in accordance with similar previous works. ⁵ , ⁶ LoEs were included in guidelines starting in the early 2000s, and we established to have at least three data points over time for attempting a trend analysis; these are the reasons why earlier guideline documents from other topics (including chronic coronary syndromes and syncope) were not included in the analysis. Finally, performing a detailed analysis of the most recent ACC/AHA guideline documents was outside the scope of our present work.

Conclusions

LoEs supporting therapeutic recommendations in contemporary ESC guidelines are generally low, particularly for topic such as HF, for which a decreasing temporal trend was noticed. Physicians should be aware of these limitations while rigorously applying these recommendations in clinical practice, and future research should aim at improving evidences supporting therapeutic interventions of doubtful efficacy. The advent of new research methodologies, a greater harmonization among documents promoted by different scientific societies, and interaction between physicians and drug companies will be key for building future fully evidence‐based recommendations and guidelines.

Conflict of interest

V.D.M. has received speaker and/or advisor fees from AstraZeneca, Daiichi‐Sankyo, Bristol‐Myers Squibb, and Bayer, outside the present work. M.C. has received speaker and/or advisor fees from Pfizer, Alnylam, AstraZeneca, Sanofi e Sanofi Genzyme, Novartis, and Vifor, outside the present work. P.A. has received speaker and/or advisor fees from AstraZeneca, Boehringer Ingelheim, Bayer, Novartis, and Vifor, outside the present work. R.F. has received personal fees from Merck Serono, Boehringer Ingelheim, Sun Pharma, Lupin, Doc Generici, Pfizer, and SPA Prodotti Antibiotici, outside the present work. He is a director of Art Research and Science S.r.l (A.R.S.1) and Scientific Director of Medical Trial Analysis. C.R. declares no conflicts of interest related to the present work. I.P. has received speaker and/or advisor fees from Biotronik, Abiomed, Terumo, Amgen, Daiichi‐Sankyo, AstraZeneca, Bayer, Medtronic, Philips, and Abbott, outside the present work. A.P.M. has received personal fees from Bayer, AstraZeneca, and Novartis, outside the present work, for the participation in committees of clinical studies.

Funding

None.

Canepa, M. , De Marzo, V. , Ameri, P. , Ferrari, R. , Tavazzi, L. , Rapezzi, C. , Porto, I. , and Maggioni, A. P. (2023) Temporal trends in evidence supporting therapeutic interventions in heart failure and other European Society of Cardiology guidelines. ESC Heart Failure, 10: 3019–3027. 10.1002/ehf2.14459.