Effect of endothelin receptor antagonists on clinically relevant outcomes after experimental subarachnoid hemorrhage: a systematic review and meta-analysis

Abstract

In clinical trials, endothelin receptor antagonists (ETRAs) reduced vasospasm but did not improve functional outcome after subarachnoid hemorrhage (SAH). We assessed the effects of treatment with ETRAs on clinically relevant outcomes in animal studies modelling SAH by performing a systematic review of the literature for controlled animal studies of ETRAs for the treatment of SAH. Primary outcomes were neurobehavioral outcomes and case fatality. Secondary outcomes were cerebral vasospasm and cerebral blood flow. Summary estimates were calculated using normalized mean difference random effects meta-analysis. We included 27 studies (55 experiments, 639 animals). Neurobehavioral scores were reported in none of the experiments, and case fatality in 8 (15%). Treatment with ETRAs was associated with a pooled odds ratio for case fatality of 0.61 (95% confidence interval (CI), 0.27 to 1.39); a 54% increase (95% CI, 39 to 69) in cerebral arterial diameter; and a 93% increase (95% CI, 58 to 129) in cerebral blood flow. We conclude that there is no evidence from animal studies that treatment with an ETRA improves clinically relevant outcomes after SAH. The reduction in cerebral vasospasm observed in animal studies is consistent with that observed in clinical trials, an effect that is not associated with better functional outcome in patients.

Introduction

Aneurysmal subarachnoid hemorrhage (SAH) is a subtype of stroke with a high risk of death or long-lasting disability.¹ Delayed cerebral ischemia (DCI) is an important cause of a poor outcome.² The only pharmaceutical treatment of proven effectiveness is oral administration of the calcium antagonist nimodipine. In clinical trials, oral nimodipine had modest effects on the risks of DCI and on the chance of a good functional outcome.³ Since the benefits of nimodipine are limited, there is a strong need for more effective drugs to prevent DCI and to improve clinical outcomes.

Until recently, endothelin receptor antagonists (ETRAs) were thought to hold great promise for improving outcome after SAH through a reduction in DCI. This was based on observations that ETRAs reduce angiographic narrowing (‘cerebral vasospasm') after SAH in animal models,⁴ and an inference that this would decrease the risk of DCI. However, in five randomized trials involving a total of 2,601 patients with SAH, ETRAs had no effects on cerebral infarction, functional outcome, or case fatality, despite a decrease in the incidence of angiographic vasospasm.⁵

We were interested to know whether this lack of efficacy against clinically relevant outcomes might have been predicted from animal SAH experiments. We therefore conducted a systematic review and meta-analysis of the efficacy of ETRAs in animal models of SAH, with particular focus on clinically relevant measures of efficacy.

Materials and methods

Systematic Search

We searched PubMed, EMBASE, and ISI Web of Knowledge on 1 January 2013 with search terms ‘subarachnoid hemorrhage', ‘subarachnoid hemorrhage', ‘subarachnoid bleeding', ‘endothelin receptor antagonists', names of individual endothelin receptor antagonists, e.g., ‘clozosentan', ‘bosentan', ‘BQ-123', ‘BQ-610', ‘BQ-788', ‘TA-0201', ‘S-0139', ‘TB-11251', ‘Ro-61-1790', ‘RO-47-0203', ‘PD-156707', ‘PD-155080', ‘PD-145065', ‘RES-701-1' and ‘FR-139317', and a combination of keywords referring to mammals used in experimental research, with singular and plural names, e.g., ‘Mouse' and ‘Mice'.⁶ Reference lists of eligible studies were screened for relevant studies. The full search strategy can be found in the online Supplementary file.

Eligibility Criteria

We included controlled studies, published in English, which tested the effects of an ETRA in animal models of SAH induced by blood injection, endovascular puncture, or intracranial administration of a blood clot where outcome was reported as neurobehavioral score, case fatality, cerebral vasospasm, or cerebral blood flow. Where the same experiment was reported in more than one publication, we included only the most recent publication. We excluded studies with insufficient data for meta-analysis, i.e., those that did not report on sample size, mean effect size, and variance.

Study Selection

Two investigators (KGL and MDIV) independently screened titles and abstracts from the electronic search for eligibility, and removed duplicates. Then, they read the full text of remaining articles to confirm eligibility and to extract data.

Data Collection

We extracted data for aspects of experimental design including measures to reduce the risk of bias and method of induction of SAH, details of drug dosage, timing, and route of administration; and the timing of outcome assessment. We extracted data to the CAMARADES Microsoft Access 2003 data manager. For continuous outcomes for each comparison between treatment and control, we extracted the number of animals in each group, the means of the outcomes and the standard deviation or standard error of these means. For case fatality, we recorded the number of deaths in each group. We included only those deaths that occurred after the start of treatment.

When neurobehavioral outcomes were assessed serially, we used only data from the final time point. We only assessed case fatality in animals in which treatment (active drug or placebo) had already been started. We extracted data for the times closest to 7 days for vasospasm and 60 minutes for cerebral blood flow. We expressed cerebral vasospasm as the percent reduction in cerebral artery diameter from baseline, and as absolute cerebral artery diameter reduction. We converted cross-sectional areas to vessel diameters in millimeters with the formulas presented in the online Supplementary file. Where data were presented in figures only we measured values using the Adobe measuring tool. In cases where a single control group served multiple treatment groups, we adjusted the size of the control group entered to the meta-analysis by dividing the size of the control group by the number of treatment groups served. We contacted authors by email to obtain unpublished or missing data.

In concordance with functional outcomes in clinical trials we considered neurobehavioral score and case fatality as the primary outcome measures. Cerebral vasospasm and cerebral blood flow were secondary outcome measures. We considered each outcome independently in the calculation of effect sizes.

Quality Assessment

We assessed each study against the CAMARADES 10-point quality checklist,⁷ consisting of (1) publication in a peer-reviewed journal; (2) control of body temperature; (3) randomized treatment allocation; (4) treatment allocation concealment; (5) blinded assessment of outcome; (6) avoidance of anesthetics with marked intrinsic neuroprotective activity (ketamine); (7) use of animals with comorbidities (hypertension or diabetes mellitus); (8) reporting of a sample size calculation; (9) statement of compliance with regulatory requirements; and (10) statement of potential conflicts of interest. Higher scores indicate a better quality.

Data Analysis

For continuous outcomes, we calculated normalized mean difference effect sizes.⁸ Differences in mortality rates were expressed as odds ratios with 95% confidence intervals (CI). We then used DerSimonian and Laird random effects modelling to calculate pooled estimates of effect size.⁹

Assessment of Heterogeneity and Publication Bias

We calculated Cochrane's Q with n−1 degrees of freedom (df)⁸ and Higgins' I² (ref. 10) to measure heterogeneity. We considered I²<50% to indicate low heterogeneity, I² between 50% and 75% moderate heterogeneity, and I²>75% large heterogeneity.¹⁰ We investigated sources of heterogeneity using stratified meta-analysis, with the chi-square distribution to assess the significance of differences between comparisons in partitioning groups. We assessed the presence of publication bias using funnel plotting¹¹, Egger regression,¹² and trim-and-fill analysis (using metatrim as an additional module for STATA).¹³

Results

We identified 205 ETRA publications, of which 27 met our eligibility criteria.^{14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40} These described the efficacy of 17 different ETRAs in 55 experiments involving 639 animals, including 363 animals in experimental groups and 276 animals in control groups (Figure 1; online Supplementary file). Four studies investigated ETRAs that had been used in clinical trials.

Figure 1.

Flowchart of study selection. SAH, subarachnoid hemorrhage.

Characteristics of Included Studies

Study characteristics are presented in Table 1. The median year of publication was 1996 (interquartile range (IQR), 1995 to 1999) and the median number of animals used per experiment was 10 (IQR, 7 to 15). None of the studies used animals with a comorbidity. In 39/55 (71%) of the experiments, SAH was induced by autologous blood injection in the cisterna magna or subdural space. The quality of the studies was generally low: median number of checklist items reported, 4 (IQR, 3 to 5). Thirteen (48%) of the studies were randomized and fourteen (52%) assessed outcomes in a blinded manner. Neurobehavioral scores were assessed in none of the experiments, and case fatality in 8 (15%).^{14, 19, 23, 27, 30, 37} None of the studies explicitly mentioned that case fatality was a predefined outcome of the study, and none reported a sample size calculation for this outcome. Of the experiments that reported case fatality, four (50%) used autologous blood clot placement,^{19, 23} one (13%) endovascular perforation,²⁷ two (25%) autologous blood injection,^{14, 37} and one (13%) donor blood injection³⁰ as a model of SAH. Also, six (75%) were randomized,^{14, 19, 23, 30} and all but one, ³⁷ had blinded outcome assessments. The median time of the final assessment of case fatality was 7 days (IQR, 6 to 7). Cerebral vasospasm was the most frequently used outcome, reported in 43 (78%) of the experiments (Table 1). Cerebral blood flow was assessed in four (7%) experiments in three studies. One study used laser-Doppler flowmetry, one a radioactive microsphere technique, and one did not report the technique used.^{16, 30, 31}

Table 1. Study characteristics.

Publications, no.	27
Experiments, no.	55
Median number of animals per experiment, median (IQR)	10 (7–15)
Species, no. (%) of experiments
Rabbit	19 (35)
Dog	17 (31)
Monkey	8 (15)
Rat	6 (11)
Mouse	4 (7)
Pig	1 (2)
Sex, no. (%) of experiments
Male	14 (25)
Female	—
Mixed	12 (22)
Not stated	29 (53)
SAH induction, no. (%) of experiments
Autologous blood injection	40 (73)
Donor blood injection	3 (5)
Autologous blood clot placement	8 (15)
Blood injection not specified	3 (5)
Endovascular perforation	1 (2)
Outcomes assessed, no. (%) of experiments
Case fatality	8 (15)
Neurobehavioral score	—
Vasospasm	43 (78)
Cerebral blood flow	4 (7)
Timing of medication administration, no. (%) of experiments
Before SAH	12 (22)
At induction of SAH	19 (35)
Within 60 minutes after SAH	13 (24)
61 minutes until 1 day after SAH	7 (13)
More than 1 day after SAH	4 (7)
Timing of outcome assessment (days; median (IQR))
Case fatality	7 (6-7)
Neurobehavioral score	N/A
Study quality (median (IQR))	4 (3–5)

Abbreviations: IQR, interquartile range; no. number; SAH, subarachnoid hemorrhage.

Treatment Effect

In studies that reported on case fatality, 12 (15%) of the 82 animals treated with an ETRA and 21 (27%) of 78 controls died before the end of the follow-up period (odds ratio, 0.61; 95% CI, 0.27 to 1.39; χ²=2.3; df=7; I²=0%) (Figure 2A). Treatment with an ETRA was associated with a 54% increase (95% CI, 39.1 to 69.0; χ² =2421; df=42; I²=98%) in cerebral arteriolar diameter (Figure 2B) and a 93% increase (95% CI, 58.4 to 128.5; χ² =2.0; df=3; I²=0%) in cerebral blood flow.

Figure 2.

Individual comparisons ranked according to the effect of endothelin receptor antagonists on case fatality (A) and cerebral vasospasm (B). The shaded gray bar represents the 95% confidence limits of the global estimate. The vertical error bars represent the 95% confidence intervals for the individual estimates.

Sources of Heterogeneity and Publication Bias

The associations between randomization or blinding and effect size for case fatality could not be assessed because the large majority of the relevant studies were randomized and blinded. The effects of ETRAs on arteriolar diameter tended to be lower in studies that reported randomization or blinded assessment of the outcome, but differences were not statistically significant (Figure 3).

Figure 3.

Influence of (A) randomization to experimental group and (B) blinding of outcome assessment on the estimate of efficacy of endothelin receptor antagonists for reduction of arterial vasospasm. The shaded gray bar represents the 95% confidence limits of the global estimate. The vertical error bars represent the 95% confidence intervals (CIs) for the individual estimates. The width of each vertical bar reflects the log of the number of animals contributing to that comparison. None of the differences were statistically significant (P>0.05).

The presence of publication bias could not be assessed for studies reporting on case fatality or cerebral blood flow because of the few of studies. Funnel plotting, Egger regression, and trim and fill suggested publication bias for studies reporting cerebral vasospasm, with 22 missing experiments in the database (online Supplementary file).

Discussion

This systematic review and meta-analysis suggests that decisions to start large randomized trials of ETRAs in patients with SAH were not supported by robust data on clinically relevant outcomes from animal studies. Although ETRAs reduced cerebral vasospasm and improved cerebral blood flow after SAH in animal models, no statistically significant benefit for case fatality was observed, and neurobehavioral outcomes were not assessed.

Most of the experimental studies used intracranial vasospasm (as determined by cerebral arteriolar diameter) as the main outcome, and this was substantially reduced by treatment with ETRAs. This is in line with the results of a recent meta-analysis of the effect of various pharmaceuticals on cerebral vasospasm in experimental SAH.⁴ In contrast to the previous meta-analysis, we also assessed the effect of ETRAs on mortality and neurobehavioral scores, and found no evidence of benefit on case fatality and no data at all on neurobehavioral scores.

Although a relationship between vasospasm, DCI, and functional outcome is biologically plausible, both the positive and negative predictive values of vasospasm for the occurrence of cerebral infarction are just around 70%.⁴¹ The futility of vasospasm as a surrogate outcome in clinical trials is supported by the fact that in clinical trials of SAH, ETRAs reduced the incidence of angiographic vasospasm, but had no effect on functional outcome or mortality.⁵ There are many other interventions that reduced vasospasm in patients, but without effect on mortality or functional outcomes in clinical trials.^{3, 42, 43} Finally, oral nimodipine has no statistically significant effect on vasospasm in patients,³ but improved functional outcomes in clinical trials.⁴⁴ For all of these reasons, we think that vasospasm is not a clinically relevant outcome in experimental SAH studies.

Comparable to the clinical setting,⁴² neurobehavioral scores and case fatality should probably be the preferred outcomes in animal studies of SAH that are designed to inform decisions to start clinical trials. This is in line with recommendations for animal studies of ischemic stroke⁴⁵ and for those of intracerebral hemorrhage.⁴⁶ However, we cannot exclude cerebral vasospasm as an important surrogate outcome in experimental studies of SAH, since changes in arterial diameter may provide insight into pathophysiologic mechanisms of brain damage. In addition, it remains to be proven whether findings from animal studies assessing these ‘clinically relevant' outcomes translate better to clinical trials than findings from animal studies that have been limited to ‘surrogate' or ‘mechanistic' outcomes such as lesion size or cerebral vasospasm. This also applies to studies using animals with comorbidities.

The studies included in this meta-analysis used intrathecal blood injection, blood clot placement, or endovascular perforation as models for SAH. The numbers of the experiments that assessed case fatality were too small to test any interaction in effect size based on the type of model.

The failure of findings in animal models of SAH to successfully translate to clinical trials may not only be explained by differences in outcomes, but also by methodological shortcomings and limited generalizability of the animal studies, as well as failure of clinical trials to replicate the conditions under which the treatment was beneficial in animal studies.^{45, 47}

Similar to our results, an earlier meta-analysis of various pharmaceuticals on cerebral vasospasm in experimental SAH found methodological limitations in most included studies and evidence of publication bias.⁴ The finding of frequently insufficient methodological quality is not unique to experimental studies of SAH, and has also been reported for animal studies on other diseases or conditions such as ischemic stroke,^{7, 48} intracerebral hemorrhage,⁴⁶ brain tumors,⁴⁹ and pain.⁵⁰ Two recent studies including data from a range of animal studies strongly suggested that publication bias is a problem associated with a substantial overestimation of the effects of treatment.^{51, 52}

A limitation of our study was that estimate of the effect of treatment on case fatality was imprecise because of the few of studies reporting this outcome. We also had insufficient power to address the effects of publication bias on case fatality.

Our study suggests that meta-analysis of the results of animal studies may detect strengths and weaknesses of published findings and should be used to inform decisions to start clinical trials. Promising treatment strategies may also be tested in high-quality, international, multicenter, preclinical ‘phase III'-type studies before moving from animal models to a clinical trial.⁵³ However, whether this strategy indeed reduces the risk of translational failure remains to be proven.

ESS, MRM, and HBvdW are members of the Multi-centre Preclinical Animal Research Team (Multi-PART; www.multi-part.org). The authors alone are responsible for the content and writing of this paper.