Randomized trials of invasive cardiovascular interventions that include a placebo control: a systematic review and meta-analysis

Abstract

Aims

The difference in the benefit of invasive cardiovascular interventions compared with placebo controls has not been analysed systematically.

Methods and results

MEDLINE and Web of Science were searched through 29 March 2020. Randomized, placebo-controlled trials of invasive cardiovascular interventions (including catheter-based interventions and pacemaker-like devices) investigating predefined primary outcomes were included. Standardized mean differences (SMD) and odds ratios were calculated for continuous and dichotomous outcomes, respectively. Meta-regression analyses were performed to assess whether estimates of treatment effects were associated with methodological characteristics of trials. Thirty trials, including 4102 patients, were analysed. The overall risk of bias was judged to be low in only 43% of the trials. Ten trials (33%) demonstrated statistically significant superiority of invasive interventions over placebo controls for the respective predefined primary outcomes. In almost half of the 16 trials investigating continuous predefined primary outcomes, the SMD between the active and placebo procedure indicated a small (n = 4) to moderate (n = 3) treatment effect of active treatment over placebo. In contrast, one trial indicated a small treatment effect in favour of the placebo procedure. In the remaining trials, there was no relevant treatment effect of active treatment over placebo. In trials with a protocol-mandated stable and symmetrical use of co-interventions, the superiority of active procedures vs. invasive placebo procedures was significantly larger as compared with trials with frequent or unbalanced changes in co-interventions (P for interaction 0.027).

Conclusions

The additional treatment effect of invasive cardiovascular interventions compared with placebo controls was small in most trials.

Graphical Abstract

Graphical Abstract.

This paper has been added by Guest Editor John Cleland, University of Glasgow, Glasgow.

See page 2569 for the editorial comment on this article (doi: 10.1093/eurheartj/ehaa558)

Introduction

In cardiovascular medicine, progress in science and technology have remarkably reduced the number of deaths from cardiovascular disease.¹ Much of this was related to the development and use of invasive interventions and surgical procedures.¹ Objective testing of new treatments starts with preclinical and first-in-man observational studies, which are ideally followed by randomized placebo-controlled trials. However, only a few trials investigating the efficacy and safety of invasive cardiovascular interventions used placebo controls.² In contrast to placebo pills, placebo procedures are invasive and are thought to be associated with a higher degree of complexity, including ethical concerns of performing a procedure conferring an immediate risk of adverse events and potential harm without potential benefit to the patient.³ As medical devices have received more public attention due to safety and efficacy issues in recent years,⁴ the US Food and Drug Administration (FDA) has called for placebo-controlled trial designs, whenever ethical and feasible.⁵ Therefore, we systematically analysed the comparative efficacy and safety of active cardiovascular interventions and placebo controls.

Methods

Search strategy and definitions

This systematic review and meta-analysis follows the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines.⁶ We searched MEDLINE and Web of Science for patient- and outcome assessor-blinded, randomized, placebo-controlled trials of invasive cardiovascular interventions. An invasive intervention was defined as a procedure during which a device was percutaneously or surgically inserted into the body and significantly modified the target-tissue. Trials using an invasive route only to administer medications (e.g. the intracoronary application of antithrombotic drugs or stem cells) or investigating cardiac resynchronization therapy/implanted cardioverter defibrillators were not considered. A placebo procedure was defined as a non-therapeutic, invasive procedure intended to mimic the active treatment as closely as possible without having a therapeutic effect beyond the placebo effect. All records through 29 March 2020 were considered, without language restrictions. Animal and paediatric studies were excluded. Detailed search terms are outlined in the Supplementary material online, Methods 1. Current clinical practice guidelines, reference lists of original articles, and review articles were hand-searched to identify further eligible trials that might have been missed using the search terms. Three reviewers (L.L., S.E., and S.S.S.) screened all abstracts independently for eligibility. Full-text articles were reviewed in duplicate by two reviewers (L.L. and S.S.S.). In the case of disagreement, a third reviewer (F.M.) was consulted, and disagreements were resolved by consensus.

Data extraction

Details on interventions, methods, patients’ characteristics, length of follow-up, outcomes, and adverse events were extracted for the active treatment and the placebo procedure group. The data underlying this article will be shared on reasonable request to the corresponding author. Although all primary efficacy and safety outcomes were extracted, only one predefined primary outcome per trial was included in the main analysis [for calculation of standardized mean differences (SMD) or odds ratios (OR)]. If a trial assessed more than one primary outcome, three reviewers (L.L., S.S.S., and F.M.) independently chose the outcome most relevant for the specific disease condition. As objective observer-reported outcomes are thought to be less prone to placebo effects than private phenomena, observer-reported outcomes were preferred, when both outcomes were available.⁷ Objective outcomes included biological measures such as blood pressure and the documentation of survival or events, whereas private phenomena were defined as subjective outcomes that were assessable by the patient only (e.g. the frequency of angina pectoris or quality of life).⁷ Results of intention-to-treat analyses were given precedence to prevent attrition bias.⁸ In crossover trials, results from the first phase were given precedence if reported separately. If data were missing, the trials’ corresponding authors were contacted.

Risk of bias assessment

Risk of bias was assessed according to the revised Cochrane risk of bias tool for randomized trials (RoB2).⁸ Two researchers (L.L. and S.E.) reviewed the publications and used the templates for randomized parallel-group and randomized crossover trials.⁸

Statistical analysis

Owing to the heterogeneity of the study populations, patients’ conditions, primary outcomes, and interventions performed, it was considered inappropriate to conduct a meta-analysis for all placebo-controlled trials. However, SMD and OR with the corresponding 95% confidence intervals were calculated for continuous (e.g. blood pressure) and dichotomous outcomes [e.g. major adverse cardiovascular events (MACE)], respectively. We standardized estimates so that positive SMDs indicated a benefit of the active intervention over placebo. Standardized mean differences were calculated by dividing the between-group difference in mean changes between baseline and follow-up by the pooled standard deviation of changes, with approximations used, as previously described.⁹ The magnitude of the SMDs was interpreted as originally suggested, with an SMD of 0.20 tentatively considered to be small, 0.50 moderate, and 0.80 large.¹⁰ We performed univariable subgroup analyses accompanied by random-effects meta-regression to test for an interaction between treatment effects and the following methodological characteristics: catheter-based intervention, pre-randomization run-in period, cut-off used to define minimal disease severity, concealment of allocation, blinding of patients, blinding of interventionalists, stable and symmetrical use of co-interventions, blinding of outcome assessors, assessment of objective outcomes, and intention-to-treat analysis (see Supplementary material online, Methods 2 for definitions of criteria for methodological quality). For these analyses, SMDs were converted to OR, as previously described.^{11  ,  12} A two-tailed P-value <0.05 was considered statistically significant. Statistical analyses were performed using STATA version 15 (StataCorp LLC, College Station, TX, USA) and GraphPad Prism version 8.2.1 (GraphPad Software, La Jolla, CA, USA).

Results

Description of trials

The search strategy identified 547 publications after duplicates were removed. Of these, 30 (5%) trials with a total of 4102 participants (median size of 97.5 patients) were eligible for the systematic review (Figure 1). Table 1 and Supplementary material online, Table S1 depict the key features of the included trials. This analysis included trials from August 2000 until March 2020. In these trials, patients were treated for angina pectoris or coronary artery disease (n = 10), hypertension (n = 9), vasovagal syncope (n = 4), chronic heart failure (n = 3), patent foramen oval suspected of causing migraine (n = 2), dysfunctional dialysis access grafts (n = 1), and carotid sinus hypersensitivity (n = 1) (Take home figure). Twenty-seven trials (90%) used a parallel-group design while three trials (10%) were designed as crossover trials.

Figure 1.

Flow diagram of trial selection (Preferred Reporting Items for Systematic Reviews and Meta-Analyses 2009).

Table 1.

Key characteristics of trials included

Author (year) Journal	Trial	Condition	Intervention (active vs. placebo)	Number of patients randomized (active/placebo)	Primary outcome
					Definition (follow-up)	Statistical superiority of active treatment over placebo
Raizner et al. (2000)13 Circulation	PREVENT	De novo/restenotic lesions	PTCA/stenting and intracoronary radiation vs. PTCA/stenting only	80/25	Incidence of MACE between baseline and 12 months	No
Stone et al. (2002)14 J Am Coll Cardiol		Refractory angina (unsuccessful percutaneous coronary intervention)	PMLR vs. coronary catheterization	71/70	Change in exercise duration at 6 months	No
Connolly et al. (2003)15 JAMA	VPS-II	Vasovagal syncope	Dual-chamber pacing (DDD) with rate-drop response vs. sensing only (ODO)	48/52	Time to first recurrence of syncope	No
Raviele et al. (2004)16 Eur Heart J	SYNPACE	Vasovagal syncope	Dual-chamber pacemaker turned on (DDD) with rate-drop response vs. pacemaker not turned on (OOO)	16/13	Recurrence of syncope	No
					Time to first recurrence of syncope	No
Serruys et al. (2004)17 Int J Cardiovasc Intervent	EUROSPAH	Stable/unstable angina or silent ischaemia	Intravascular ultrasound after stenting vs. stenting only	202/201	In-stent late lumen loss at 6 months	No
Salem et al. (2004)18 Am J Cardiol	BELIEF	Stable angina	PMLR vs. laser turned on, but no procedure	40/42	Improvement of ≥1 CCS angina classes at 12 months	Yes
Leon et al. (2005)19 J Am Coll Cardiol	DIRECT	Refractory angina	PMLR vs. laser turned on, but no procedure	98/102	Change in exercise duration at 6 months	No
Geiger et al. (2006)20 Strahlenther Onkol	EVEREST	Coronary artery disease	PTCA/BMS and intracoronary radiation vs. PTCA/BMS	21/11	Composite clinical outcomea at 16 months	No
Syeda et al. (2006)21 Radiother Oncol	REGARD	Coronary artery disease in diabetic patients	PTCA/stenting and intracoronary radiation vs. PCTA/stenting	45/44	Incidence of thrombosis/MACE at 9 months	No
					Late lumen loss at 9 months	Yes
					Restenosis >50% at 9 months	Yes
Reynen et al. (2006)22 Coron Artery Dis		Coronary in-stent restenosis	PTCA and intracoronary radiation vs. PTCA	78/78	Diameter of stenosis at 6 months	No
					Re-restenosis rate at 6 months	Yes
Misra et al. (2006)23 Kidney Int	BRAVO I	Dysfunctional dialysis access graft	PTA and endovascular radiation vs. PTA only	14/11	Angiographic target lesion primary patency at 6 months	Yes
Dowson et al. (2008)24 Circulation	MIST	PFO and migraine with aura	Percutaneous PFO occlusion vs. skin incision	74/73	Cessation of migraine headache 3 through 6 months after randomization	No
Parry et al. (2008)25 Heart		Falls attributed to carotid sinus hypersensitivity	Dual-chamber pacing (DDD) with rate-drop response vs. sensing only (ODO)	34 (crossover)	Number of falls at 6 months	No
Bisognano et al. (2011)26 J Am Coll Cardiol	Rheos Pivotal Trial	Resistant hypertension	BAT device implantation and turned on vs. BAT device not turned on	181/84	Composite efficacy outcomeb	No
Brignole et al. (2012)27 Circulation	ISSUE-3	Neurally mediated syncope	Dual-chamber pacing (DDD) with rate-drop response vs. sensing only (ODO)	38/39	Recurrence of syncope between baseline and 24 months	Yes
Bhatt et al. (2014)28 N Engl J Med	SYMPLICITY HTN-3	Resistant hypertension	Catheter-based RDN (monoelectrode radiofrequency catheter) vs. renal angiography only	364/171	Change in office SBP at 6 months	No
Zannad et al. (2015)29 Eur Heart J	NECTAR-HF	HFrEF	Implantation and vagal nerve stimulator system turned on vs. implantation only	59/28	Change in left ventricular systolic diameter at 6 months	No
Verheye et al. (2015)30 N Engl J Med	COSIRA	Refractory angina	Coronary sinus reducer vs. coronary angiography	52/52	Improvement of ≥2 CCS angina classes at 6 months	Yes
Desch et al. (2015)31 Hypertension	Leipzig RSD	Resistant hypertension	Catheter-based RDN (monoelectrode radiofrequency catheter) vs. renal angiography only	32/35	Change in 24 h-ambulatory SBP at 6 months	No
Mathiassen et al. (2016)32 J Hypertens	ReSET	Resistant hypertension	Catheter-based RDN (monoelectrode radiofrequency catheter) vs. catheter in situ and radiograph scan	36/33	Change in ambulatory daytime SBP at 3 months	No
Beige et al. (2017)33 J Hypertens		Resistant hypertension	BAT device turned on vs. BAT device turned off	17 (crossover)	Intraindividual increase in office SBP ≥35 mmHg while BAT device turned off	No
Al-Lamee et al. (2018)34 Lancet	ORBITA	Symptomatic angiographically significant (≥70%) non-occluded lesion in a single vessel	PTCA/DES vs. coronary angiography	105/91	Change in exercise duration at 6 weeks	No
Tobis et al. (2017)35 J Am Coll Cardiol	PREMIUM Migraine	PFO and migraine with/without aura	Percutaneous PFO closure vs. right heart catheterization	123/107	50% reduction of monthly number of migraine attacks during months 10 through 12 compared with baseline	No
Baron-Esquivias et al. (2017)36 J Am Coll Cardiol	SPAIN	Vasovagal syncope	Dual-chamber pacing (DDD) with CLS vs. placebo pacing (DDI)	46 (crossover)	Reduction of syncopal episodes ≥50% at 12 months	Yes
					Time to first recurrence of syncope	Yes
Feldman et al. (2018)37 Circulation	REDUCE-LAP HF I	HFpEF (EF ≥40%) and elevated left atrial pressure	Transcatheter interatrial shunt device vs. intracardiac echocardiography	22/22	Difference in exercise PCWP at 1 month	Yes
Kandzari et al. (2018)38 Lancet	SPYRAL HTN-ON MED	Mild-to-moderate hypertension	Catheter-based RDN (multi-electrode radiofrequency catheter) vs. renal angiography only	38/42	Change in 24 h-ambulatory blood pressure at 6 months	Yes
Azizi et al. (2018)39 Lancet	RADIANCE HTN SOLO	Mild-to-moderate uncontrolled or controlled (≤2 drugs) hypertension	Catheter-based RDN (ultrasound-based catheter) vs. renal angiography only	74/72	Change in daytime SBP at 2 months	Yes
Witte et al. (2019)40 JACC Heart Fail	REDUCE FMR	Functional mitral regurgitation	Catheter-based mitral annuloplasty vs. coronary sinus angiography only	87/33	Change in mitral regurgitant volume at 12 months	Yes
Weber et al. (2020)41 JACC Cardiovasc Interv	REDUCE HTN: REINFORCE	Mild-to-moderate hypertension in the absence of antihypertensive medication	Catheter-based RDN (bipolar radiofrequency catheter) vs. renal angiography only	34/17	Change in 24 h-ambulatory blood pressure at 2 months	No
Böhm et al. (2020)42 Lancet	SPYRAL HTN-OFF MED Pivotal (including pilot phase)	Mild-to-moderate hypertension in the absence of antihypertensive medication	Catheter-based RDN (multi-electrode radiofrequency catheter) vs. renal angiography only	166/165	Change in 24 h-ambulatory blood pressure at 3 months	Yes

BAT, baroreceptor activation therapy; BMS, bare metal stent; CCS, Canadian Cardiovascular Society; CLS, closed-loop-stimulation; DES, drug-eluting stent; EF, ejection fraction; HFpEF/HFrEF, heart failure with preserved/reduced ejection fraction; MACE, major adverse cardiovascular events; PCWP, pulmonary capillary pressure; PFO, patent foramen oval; PMLR, percutaneous myocardial laser revascularization; PTA, percutaneous transluminal angioplasty; PTCA, percutaneous transluminal coronary angioplasty; RDN, renal denervation; SBP, systolic blood pressure.

^aComposite of death, myocardial infarction, repeat target lesion or percutaneous revascularization, and coronary artery bypass surgery.

^bComposite of (i) blood pressure responder rate at 6 months, (ii) sustained blood pressure-lowering response at 12 months, (iii) procedure, (iv) BAT, and (v) device safety.

Take home figure.

This systematic review and meta-analysis analyses the comparative efficacy and safety of invasive cardiovascular interventions compared with invasive placebo procedures and the interactions between treatment effects and methodological characteristics.

Risk of bias

Figure 2 depicts a summary of the risk of bias, with details on the rationales for judgments provided in the Supplementary material online, Results. The overall risk of bias was judged to be low in 13 trials (43%).^{16  ,  24  ,  28  ,  30–32  ,  34  ,  36  ,  41  ,  42} There were some concerns and a high risk of bias in 14 (47%) and 3 (10%) trials, respectively. The interventionalist and outcome assessors were adequately blinded in only 10 (33%) and 25 (83%) trials, respectively. The underlying reasons for judging three trials to be at high risk of bias were the pooling of data of an unblinded pilot phase and the randomized trial phase,²⁰ insufficient blinding of outcome assessors due to device-induced artefacts seen during the echocardiographic assessment of the primary outcome⁴⁰ and some concerns for multiple domains of bias.²⁵

Figure 2.

Risk of bias assessment of all included trials. Risk of bias was assessed according to the revised Cochrane risk of bias tool for randomized trials (RoB2).⁸

Primary outcomes

In total, the included trials evaluated 35 predefined primary outcomes, including 15 dichotomous (e.g. MACE) and 20 continuous (e.g. change in blood pressure) outcomes. Four trials assessed coprimary outcomes.^{16  ,  21  ,  22  ,  36} In 10 trials (33%), the null hypotheses were rejected for all predefined primary outcomes.^{18  ,  23  ,  27  ,  30  ,  36–40  ,  42} Of the 30 outcomes included in the main analysis, the majority of the trials assessed objective outcomes (n = 26, e.g. change in blood pressure or MACE) while four trials used patient-reported outcomes (private phenomena, e.g. the severity of angina pectoris). Active treatment demonstrated significant superiority over placebo procedures in 8/26 (31%) and 2/4 (50%) of the objective and patient-reported outcomes, respectively.

In 5/16 (31%) trials assessing a continuous primary outcome, the active treatment was significantly superior to placebo (Figure 3). The SMD between the active and placebo procedure was at most small (SMD 0.2–0.5, n = 5) to moderate (SMD 0.5–0.8, n = 3). Figure 4 presents OR for the trials reporting dichotomous primary outcomes. Of these, 4/14 (29%) trials showed the superiority of active treatment over placebo for their primary outcome. One trial had not observed any primary outcome events.³³

Figure 3.

Comparison of the treatment effect of active vs. placebo procedures for continuous primary outcomes. Standardized mean differences were standardized, so that positive values indicated a benefit of the active intervention over placebo. LV, left ventricular; PCWP, pulmonary capillary wedge pressure; PTCA + DES, percutaneous transluminal coronary angioplasty and implantation of a drug-eluting stent; SBP, systolic blood pressure.

Figure 4.

Comparison of the treatment effect of active vs. placebo procedures for binary primary outcomes. ^aComposite of death, myocardial infarction, repeat target lesion or percutaneous revascularization, and coronary artery bypass graft. ^bResponder rate was defined as a blood pressure drop ≥10 mmHg in systolic office blood pressure as a part of a composite outcome. ^cResponse was defined as an increase in office systolic blood pressure >35 mmHg while the baroreflex activation therapy device was turned off. BP, blood pressure; CCS, Canadian Cardiovascular Society; MACE, major adverse cardiovascular events; PFO, patent foramen oval.

Subgroup analyses by methodological characteristics of trials

Figure 5 shows the results of subgroup analyses according to the methodological characteristics of trials (see Supplementary material online, Table S2). In trials with a protocol-mandated stable and symmetrical use of co-interventions, the average difference in effects between active and placebo procedures was larger compared to trials with frequent and/or unbalanced changes in co-interventions (P for interaction 0.027). For the remaining methodological characteristics, there were only minor variations between subgroups of trials.

Figure 5.

Subgroup analyses by methodological characteristics of trials. Subgroup analyses accompanied by random-effects meta-regression were performed to test for an interaction between treatment effects and methodological characteristics of trials (see Supplementary material online, Methods 2). For these analyses, standardized mean differences were converted to odds ratios.

Safety outcomes

Twenty-six trials (87%) provided data on predefined safety outcomes (Supplementary material online, Table S3). Of these, six trials (23%) reported no adverse events or safety outcomes in the active treatment or placebo group, whereas nine trials (35%) reported more adverse events following placebo procedure than active treatment but did not report further statistical analyses.^{32  ,  37}

Discussion

In total, 30 placebo-controlled trials analysed 35 predefined primary outcomes. In only 10 trials, the null hypothesis was rejected for all predefined primary outcomes. In trials assessing continuous primary outcomes, the effectiveness, defined as the SMD between the active and placebo procedure, was at most small to moderate. More than two-thirds of the trials evaluating dichotomous outcomes failed to show the superiority of active over placebo treatment. Taken together, this suggests that the therapeutic efficacy of some active experimental treatments in interventional cardiology may be smaller than generally assumed. Subgroup analyses, according to methodological characteristics, indicated that average treatment effects were larger in trials with stable and symmetrical use of co-interventions.

A placebo is generally defined as a substance or treatment of no intended therapeutic value, although it can exhibit relevant effects.² The placebo effect refers to a clinical benefit attributable to the interaction of the patient with the healthcare system and underlies complex psychological and neurobiological mechanisms.⁴³ There are strong indications that there is not one mechanism of the placebo effect, but many, depending on the physiological system involved, the medical condition and its severity as well as the placebo’s nature.⁴³ The appropriate placebo control in interventional cardiology is thought to be an invasive placebo, mimicking the interventions of the active treatment closely. It has indeed been suggested that the method of treatment delivery and the invasiveness of the placebo procedure correlates with its effectiveness.^{44  ,  45} In a meta-analysis investigating studies of migraine prophylaxis, placebo surgery was associated with higher responder rates (58%) than placebo acupuncture (38%) or oral pharmacological placebos (22%), respectively.⁴⁴ Similarly, the response to placebo pills in the treatment of Parkinson’s disease was minimal, whereas the response to placebo surgery large.⁴⁵

In the recent ORBITA trial, patients with stable angina and angiographically significant non-occluded lesions ≥70% were randomly allocated to percutaneous coronary intervention (PCI) with a drug-eluting stent or a placebo procedure (coronary catheterization while the patient was sedated).³⁴ In contradiction to randomized controlled trials which indicated that subjective outcomes, such as the severity of angina, was reduced following PCI compared with optimal medical therapy alone,^{46  ,  47} there was no difference in the primary efficacy outcome of exercise duration at 6 months, an objective but patient-dependent outcome, between patients who underwent PCI compared with a placebo procedure.³⁴ Several potential reasons have been discussed, which include methodological issues, including insufficient power due to a higher than expected standard deviation and a smaller than expected treatment effect. Importantly, the commonly observed symptomatic improvement from PCI may be encouraged by the potential power of placebo procedures.² The selection of the primary outcome appears essential, as patient-reported outcomes, in general, were more susceptible to placebo effects compared with observer-reported outcomes.⁴⁸ In this analysis, the null hypothesis for patient-reported (e.g. angina pectoris) and objective observer-reported outcomes (e.g. change in blood pressure) was rejected in 50% and 31% of the trials, respectively.

Blood pressure, as an objective primary efficacy outcome, was assessed in seven placebo-controlled trials investigating device-based therapies for hypertension. Early uncontrolled trials and registries of device-based therapies for hypertension documented large falls in blood pressure.⁴⁹ In the SYMPLICITY HTN-3 trial, the first placebo-controlled trial investigating renal denervation in patients with severe, resistant hypertension, blood pressure was reduced following active and placebo treatment without significant between-group differences.²⁸ The same observation was documented in a trial investigating baroreflex activation therapy in severe, resistant hypertension.²⁶ Several potential factors, such as inadequate patient selection, changes in antihypertensive medications after randomization, lifestyle changes, and variation in adherence to medication may have contributed to the significant blood pressure drop following placebo treatment.⁵⁰ Recently published trials, however, which minimized numerous issues identified in prior trials, provided proof of principle for the efficacy of renal denervation in both the presence and absence of antihypertensive medications.^{3  ,  38  ,  39  ,  42} Interestingly, in these well-controlled, rigorously executed trials no significant placebo effect on blood pressure was noticed, indicating that the use of placebo procedures per se does not eliminate other sources of bias that could result in both an underestimation and overestimation of treatment effects. In our analysis, subgroup analyses according to the methodological characteristics suggested that the treatment effect was largest in trials with stable and symmetrical use of co-interventions in active and placebo groups.

Objective testing of new devices and technologies in randomized trials with an appropriate control intervention is desirable wherever feasible. The fact that half of the trials eligible for the present analysis were published in 2015 or later indicates a growing awareness of the need to minimize the risk of bias in interventional cardiology. The use of a placebo procedure, however, adds complexity. Not only the patients’ blinding but also blinding of interventionalists and outcome assessors are necessary as incomplete blinding may introduce performance and detection bias.^{51  ,  52} Especially when investigating invasive interventions, it can be challenging to assure successful patient’s and physician’s blinding. Blinding of outcome assessors and treating physicians is important to reduce bias arising due to asymmetrical co-interventions (deviations from intended intervention) and bias in measuring the outcome.⁸ In five trials (17%) included herein, it remained unclear whether or not the outcome assessors were adequately blinded.^{20–23  ,  40} In a placebo-controlled trial assessing the effects of chronic vagal nerve stimulation for the treatment of heart failure with reduced ejection fraction, the majority of patients in the active treatment group (77%) correctly guessed their randomization assignment due to tickling sensations and other signs and symptoms during titration to the highest comfortable output. Patient’s blinding is in particular important when investigating private phenomena, whereas objective observer-reported outcomes are thought to be less prone to placebo effects.⁷ Blinding success is often underreported or, if reported, unsatisfactorily low.⁵³

The advantages of patient- and outcome assessor-blinded, placebo-controlled trials in reducing bias and investigating the specific treatment effect of invasive interventions over and above placebo effects are undeniable but exposing patients to potential risk by subjecting them to an invasive placebo procedure can raise ethical concerns. Therefore, the potential benefit of interventions has to be carefully weighed against the risks of a convincing placebo procedure, and there must be a sense of equipoise between active intervention and placebo control due to conflicting or weak evidence on the effectiveness of the intervention. Safety concerns are frequently raised as an argument against the use of invasive placebo procedures. For many of the trials included in this analysis, the risk of adverse events following placebo was relatively low, thus not necessarily supporting ethical arguments against placebo interventions. Taken together, this indicates that the benefits of including a placebo group need to be carefully weighed against the risk on a case-by-case basis.

Limitations

Some limitations of our analysis need to be acknowledged. First, assessing the risk of bias was problematic, as, especially in older trials, methods for blinding were only vaguely described. Second, we cannot exclude that regression to the mean might have contributed to the change of an outcome of interest. However, as we only included randomized controlled trials, regression to the mean is assumed to be equally distributed between active and placebo groups.^{44  ,  54} Third, it is likely that the failure to reject the null hypothesis in some trials was due to a lack of power to detect minimal clinically important differences between groups, particularly if the effect size used for sample size considerations was unrealistically large. Fourth, even though there are more than 200 randomized trials, which compared placebo against non-interventional controls, that did not receive a placebo,⁴⁸ none of the trials included in our analysis used a non-interventional control; hence, the true magnitude of the placebo effect is impossible to estimate.

Conclusion

The SMD between active and placebo procedures was at most small to moderate, which underlines the influence of non-specific effects on trial outcomes and an overestimation of the clinical efficacy of interventions in many circumstances. For most trials, the risk of adverse events following placebo was relatively low. Finally, our analysis suggests that treatment effects were larger in trials with protocol-mandated stable and symmetrical use of co-interventions after randomization, which highlights the significance of diligent planning and execution of placebo-controlled trials.

 

Conflict of interest: S.E. has received scientific support and speaker honoraria from Medtronic and ReCor Medical. S.S.S. has received speaker honoraria from Pfizer. W.W. has received a research grant and honoraria from MicoPort. T.F.L. has received educational and research grants from Abbott/St. Jude, Biotronik, and Medtronic. E.R.E. has active research programmes and grant support from Boston Scientific, Cardiatis, Edwards Life Sciences, Medtronic and is a founder of BioRest and Panther therapeutics. E.R.E. is funded in part by a grant from the US National Institutes of Health (R01 GM 49039). M.B. has received honoraria for lectures and scientific advice from Abbott, Astra Zeneca, Boehringer Ingelheim, Medtronic, Servier, and Vifor. P.J. serves as unpaid member of the steering group of trials funded by Astra Zeneca, Biotronik, Biosensors, St. Jude Medical, and The Medicines Company, has received research grants to the institution from Astra Zeneca, Biotronik, Biosensors International, Eli Lilly, and The Medicines Company, and honoraria to the institution for participation in advisory boards from Amgen, but has not received personal payments by any pharmaceutical company or device manufacturer. F.M. is supported by Deutsche Gesellschaft für Kardiologie (DGK) and Deutsche Forschungsgemeinschaft (SFB TRR219) and has received scientific support and speaker honoraria from Bayer, Boehringer Ingelheim, Medtronic, and ReCor Medical. All other authors declared no conflict of interest.