Factors affecting the apparent efficacy and safety of tissue plasminogen activator in thrombotic occlusion models of stroke: systematic review and meta-analysis

Abstract

Thrombolysis with recombinant tissue plasminogen activator (rtPA) improves outcome in animal models of stroke and in clinical trial, but is associated with increased intracranial hemorrhage. Here, we explore the impact of biologic and experimental design factors on efficacy and bleeding. We conducted a systematic review of studies describing the effect of tPA in thrombotic occlusion models of ischemic stroke followed by random effects meta-analysis, meta-regression, and trim and fill. We identified 202, 66, 128, and 54 comparisons reporting infarct volume, neurobehavioral score, hemorrhage, and mortality, respectively. The rtPA reduced infarct volume by 25.2% (95% confidence interval=21.8 to 28.6, 3388 animals), improved neurobehavioral score by 18.0% (12.6% to 23.3%, n=1243), increased the risk of hemorrhage (odds ratio=1.71, 1.42 to 2.07, n=2833) and had no significant effect on mortality (odds ratio=0.82, 0.62 to 1.08, n=1274). There was an absolute reduction in efficacy of 1.1% (0.7% to 1.4%) for every 10 minutes delay to treatment. Cumulative meta-analysis showed that the estimate of efficacy fell as more data became available. Publication bias inflated efficacy by 5.1% (infarct volume) and 8.1% (neurobehavioral score). This data set was large enough to be adequately powered to estimate with precision the impact of biologic and experimental factors on the efficacy and safety of rtPA.

Introduction

The difficulties encountered in the development of effective neuroprotective agents for stroke have raised questions about the design and conduct of studies to evaluate new agents in animals and humans (Fisher et al, 2005; Dirnagl, 2006). Although it has been suggested that knowledge of tissue plasminogen activator (tPA) efficacy in human myocardial infarction was sufficient to warrant its testing in human clinical trials in stroke (Mori et al, 1992), the nature of ischemic brain injury and the potential for further injury at the time of reperfusion provides a powerful argument for testing tPA in animal models of focal cerebral ischemia. It has been suggested that animal experiments may not model human stroke with sufficient fidelity to be useful in preclinical drug development (Macleod et al, 2005). Although 503 of 835 candidate stroke drugs showed efficacy in animal models of focal cerebral ischemia (O'Collins et al, 2006), only tPA has been proven to be sufficiently effective to have secured a place in routine clinical practice (albeit in highly selected patients).

A meta-analysis of data from six randomized clinical trials (Wardlaw et al, 2009) confirmed that tPA is of greatest benefit when given within 3 hours of symptom onset and is effective to at least 4.5 hours. tPA therapy administered within 3 hours significantly reduced the proportion of patients who were dead or dependent (modified Rankin Score (mRS) 3 to 6) at three to six months (OR 0.64, 95% CI 0.50 to 0.83). Specifically, the NINDS study reported that the increase in patients surviving with an mRS of 0 to 1 was sustained for at least 12 months (NINDS rt-PA Stroke Study Group, 1995). This was in spite of an increased incidence of intracranial hemorrhage (OR 3.40, 95% CI 1.48 to 7.84). However, it is not clear whether the treatment might be of net benefit in a wider population (i.e., after 4.5 hours, in elderly patients and in patients with hypertension). In addition, the relative efficacy of intravenous versus intra-arterial treatment is unclear. A number of clinical trials are underway to address these questions (Lindsberg and Mattle, 2006; Khatri et al, 2008).

The development of tPA therapy for human stroke has proceeded hand in hand with this laboratory work and therefore this is clearly not a simple case of the sequential demonstration of efficacy in animals and then in humans. Nonetheless, the presence of two large data sets reporting efficacy in animals and humans, respectively, provides a unique opportunity to understand better the process of translation.

We have previously published a brief overview of data testing the efficacy of tPA in all species in temporary, permanent, and thrombotic occlusion models published up to 2005 (Perel et al, 2007). The principal purpose of that analysis was to explore concordance between animal and clinical studies for an intervention known to be effective in humans. However, there was substantial heterogeneity in the data set, which included all forms of occlusion.

Here, we report an updated systematic review and meta-analysis of the efficacy of tPA treatment in thrombotic occlusion models. Our aims were to provide a robust summary of the efficacy of tPA in experimental models and second, where comparable data were available to provide a comparison between animal and human data.

These include examining the biologic factors that influence efficacy and safety and the concordance with what is known about these factors in humans; and to examine items of internal and external validity (Crossley et al, 2008) and to provide empirical evidence of their impact on outcome. In addition to providing further insight into the characteristics of the protective efficacy of tPA, a deeper understanding of this literature is likely to inform the successful development of other candidate stroke drugs.

The tPA is the only stroke therapy with concordance between animal and human studies for which there is a large enough body of animal evidence to allow sufficient power for subgroup analyses and to assess interactions. This data set is therefore large enough to be adequately powered to estimate with precision the impact of biologic and experimental factors on the efficacy and safety of recombinant tPA (rtPA).

Materials and methods

We use normalized mean difference random effects meta-analysis to account for the observed heterogeneity in previous analyses; to assess the impact of various determinants on outcome through systematic review, we have performed meta-regression on these data; and we have estimated the impact of publication bias using trim and fill.

Search Strategy

(1) Electronic search (December 2007) of Pubmed, EMBASE, BIOSIS using the search terms ((thrombolysis) OR (tissue plasminogen activator) OR (tPA) OR (t-PA) OR (rt-PA) OR (rtPA) OR (desmoteplase) OR (alteplase)) AND ((stroke) OR (ischemia) OR (cerebrovascular) OR (middle cerebral artery) OR (MCA) OR (ACA) OR (anterior cerebral artery) OR (MCAO)) AND Animals NOT (coronary OR myocardia^*); (2) hand searching of abstracts of 3rd to 5th World Stroke Congresses, European Stroke Conference (from 2nd meeting/1992 onwards), International Stroke Conference (from 25th meeting/2000 onwards; previously in BIOSIS), conferences of the International Society for Cerebral Blood and Metabolism (from 16th meeting/1993 onwards); and (3) requests to authors of publications identified above for other published or unpublished data.

Criteria for Including Studies

Two investigators (CB, MM) independently assessed the titles and abstracts identified by these searches and obtained copies of publications, which described controlled studies of tPA given in thrombotic models of focal cerebral ischemia, where tPA was administered by any route and outcome was compared with animals receiving placebo or no tPA. The tPA includes its analogues (tenecteplase, SUN-9216, monteplase, and palmiteplase). Disagreements about the selection of studies were resolved in discussion with a third investigator (ES).

Outcome Data Extracted from Publications

The primary outcome measure extracted from publication was infarct area or volume (determined histologically or by cross-sectional imaging); we also extracted data on neurobehavioral score, on intracranial hemorrhage, including petechial hemorrhage and hematoma, and on mortality.

Methods of the Review

Quality of studies

We assessed study quality against the CAMARADES 10-item study quality checklist (Macleod et al, 2004) that comprises (1) publication in peer-reviewed journal, (2) statement of control of temperature, (3) randomization to treatment or control, (4) masked induction of ischemia (i.e., concealment of treatment group allocation at time of induction of ischemia), (5) masked assessment of outcome, (6) avoidance of anesthetics with marked intrinsic neuroprotective properties, (7) use of animals with hypertension or diabetes, (8) sample size calculation, (9) statement of compliance with regulatory requirements, and (10) statement regarding possible conflicts of interest.

Data extraction

From each source we identified individual comparisons where outcome was measured in a group of animals receiving a specified dose(s) of tPA at a specified time(s) and compared with outcome in a control group. Where the treatment group received more than one intervention this was recorded. For each comparison and for each treatment and control group, we extracted data for number per group, mean outcome, and its s.d. Where an outcome was measured serially, only the final measure was used. Where more than one outcome measure was reported (infarct volume, neurobehavioral score, hemorrhage, mortality) all were recorded. Where data were presented only graphically, we contacted the authors to seek the raw data; and where this was not available, we estimated values by measurement from the publication. Two reviewers extracted data independently, and differences were resolved by discussion.

We also collected other relevant data including anesthetic used, time of outcome measurement, and method of induction of ischemia, as well as the individual component items of the quality checklist above.

Analysis

For continuous variables (infarct volume, neurobehavioral score), we normalized data to outcome in the control group, that is:

Effect sizes were combined to give an overall estimate using weighted mean difference with a random effects model. Where a single control group served multiple treatment groups, the size of the control group entered to the meta-analysis was adjusted by division by the number of treatment groups served. For cumulative meta-analysis, we assessed the effect of adding studies sequentially according to the date of publication on the effect of tPA using cumulative meta-analysis where studies are also weighted according to their variance.

Exploratory Analyses of Factors Modifying Effect of Tissue Plasminogen Activator

We assessed the impact on the efficacy of tPA of a number of variables using both, stratified meta-analysis and meta-regression. Variables included drug dose; time of administration; study quality; individual components of study quality checklist; the presence of cotreatments; anesthetic used; use of mechanical ventilation; gender of animal used; outcome measure used; and time interval from treatment to final assessment of outcome. The significance of differences between n groups was assessed by partitioning heterogeneity and by using the χ² distribution with n−1 degrees of freedom (df). To allow for multiple comparisons, we adjusted our significance level using Bonferroni correction; P<0.003 for infract volume and neurobehavioral score and P<0.006 for hemorrhage and mortality.

Power

Previous observations show modeling (using bootstrapping) of stratified meta-analyses of animal data (Wu, 2006); this suggests that meta-analyses of data from 200 experiments stratified in two groups would have 80% power to detect a 10% absolute difference in reported outcome at our prespecified 0.001 level when the groups were of equal size; 67% power to detect a 10% absolute difference in reported outcome at our prespecified 0.001 level when there were 150 experiments in one group and 50 in the other; and 73% power to detect a 15% absolute difference in reported outcome at our prespecified 0.001 level when there were 180 experiments in one group and 20 in the other. This modeling suggested that, where the number of contributing experiments was substantially lower, the power was reduced accordingly.

Publication Bias

Publication bias was assessed with a funnel plot and with the Egger regression method (Sterne et al, 2001) and the effect size was adjusted for the presence of this bias using ‘trim and fill' (Duval and Tweedie, 2000) enabled in STATA.

Results

We identified 104 publications that met our inclusion criteria and were included in this review (Supplementary Appendix 1). These 104 publications described 450 comparisons; 202 in 3388 animals reported data as infarct volume, 128 in 2833 animals as hemorrhage, 66 in 1243 animals as a neurobehavioral score and 54 in 1274 animals as mortality. For infarct volume, an analysis excluding analogues of tPA is available as an online supplement (Supplementary Appendix 2).

Study Quality and Publication Bias

The median quality score was 4 out of a possible 10 (interquartile range 3 to 5, Supplementary Appendix 3). There was a significant association between study quality and outcome for infarct volume and neurobehavioral score. For both infarct volume and neurobehavioral, high-quality studies give low estimates of efficacy and vice versa (Figure 1). Specifically, masked assessment of outcome was associated with higher estimates of efficacy for neurobehavioral score and higher observed odds of hemorrhage (not shown). Masked assessment of outcome and random allocation were associated with higher odds of death in the control group (not shown). The funnel plot analysis showed that there was an excess of small studies overestimating the true efficacy suggesting publication bias, which was confirmed by Egger regression (P<0.0001). Trim and Fill further confirmed this and estimated the impact of this bias to be an absolute overstatement of efficacy of 5.1% for infarct volume and 8.1% for neurobehavioral score (Table 1).

Figure 1.

Effect of reported study quality on, the estimate of efficacy on (A) infarct volume and (B) neurobehavioral score. The shaded gray bar represents the 95% confidence limits of the global estimate. The vertical error bars represent the 95% confidence intervals for the individual estimates. The width of each bar reflects the log of the number of animals contributing to that comparison. Stratification by study quality accounts for a significant proportion of the heterogeneity observed between studies.

Table 1. Scale of publication bias.

Outcome measure	Reported effect size (95% CI)	Bias with egger regression	Bias with METATRIM	Additional % studies considered ‘missing'	METATRIM adjusted effect size (95% CI) (Sena et al, 2010)	Absolute overstatement of efficacy (%)	Relative overstatement of efficacy (%)
Infarct volume	25.2% (21.9–28.6)	+	+	27	20.2% (16.7–23.7)	5.1	25
Neurobehavioral score	18.0% (12.6–23.3)	+	+	15	9.89% (4.13–15.7)	8.11	82

CI, confidence interval.

Efficacy

Overall, tPA reduced infarct volume by 25.2% (95% confidence interval=21.8% to 28.6% 202 comparisons; Figure 2A), improved neurobehavioral score by 18.0% (12.6% to 23.3% 66 comparisons; Figure 2B); was associated with increased frequency of hemorrhage (odds ratio=1.71, 1.42 to 2.07; 128 comparisons; Figure 2C); and had no significant effect on odds of death (odds ratio=0.82, 0.62 to 1.08; 54 comparisons; Figure 2D). There was substantial between-study heterogeneity for the analysis of infarct volume (χ²=674, df=201, P<0.001) and neurobehavioral score (χ²=225, df=65, P<0.001) but not of hemorrhage (χ²=131, df=127, P=0.39) or of mortality (χ²=54, df=53, P=0.41). This heterogeneity suggests that these results must be interpreted with caution.

Figure 2.

Individual comparison ranked according to their effect on (A) infarct volume, (B) neurobehavioral score, (C) odds of hemorrhage, and (D) odds of mortality. The shaded gray bar represents the 95% confidence limits of the global estimate. The vertical error bars represent the 95% confidence intervals for the individual estimates.

In cumulative meta-analysis, the overall effect on infarct volume fell as experiments were sequentially included (Figure 3). This analysis suggests the estimate of efficacy was stable from around 2001, after data from ∼1500 animals had been included.

Figure 3.

Cumulative meta-analysis; studies included by publication date and the effect of this on the estimate of efficacy.

Regression Analysis

Infarct volume

Meta-regression identified three factors (the time of tPA administration; and method of occlusion; statement of a sample size calculation) to account for 32% of between-study variance (adjusted R²=0.323, τ²=232.5). There was an absolute reduction in efficacy of 1.1% (0.7% to 1.4%) for every 10 minutes delay to treatment. Studies that did not report a sample size calculation (189/204) estimated the absolute difference in efficacy to be 21.1% (5.93% to 36.3%) higher than those that did report this measure. Induction of occlusion via photochemical thrombosis was associated with an absolute increase in efficacy of 23.8% (13.8% to 33.8%) compared with autologous embolism.

Neurobehavioral score

For neurobehavioral score a regression model based on three factors (the neurobehavioral test used, the presence of comorbidity and the anesthetic used) explained almost three quarters of the between-study variance (adjusted R²=0.743, τ²=95.8). Studies that used healthy animals report estimates of efficacy 43.2% higher (19.1% to 67.4%) (absolute difference) than those that did not. Pentobarbital anesthesia was associated with an absolute increase in efficacy of 20.0% (0.99% to 38.9%) compared with halothane.

Stratified Analysis

Efficacy was highest where it was measured within the first 3 days (infarct volume; Figure 4A, χ²=22.4 df=2, P<0.003, neurobehavioral score; Figure 4B, χ²=56.1 df=2, P<0.003). Assessment within 24 hours was associated with higher odds of death (χ²=13.4 df=2, P<0.006; Figure 4C).

Figure 4.

Effect of time of outcome assessment on (A) infarct volume, (B) neurobehavioral score, and (C) odds of mortality. The shaded gray bar represents the 95% confidence limits of the global estimate. The vertical error bars represent the 95% confidence intervals for the individual estimates. The size of each point reflects the log of the number of animals contributing to that comparison. Each stratification accounts for a significant proportion of the heterogeneity observed between studies.

In studies that reported the use of hypertensive animals (all spontaneously hypertensive rats), tPA had no significant effect on either infarct volume (Figure 5A: χ²=27.1, df=1, P<0.003) or neurobehavioral score (χ²=49.8, df=1, P<0.003; Figure 5B). For both infarct volume and neurobehavioral score, tPA was most effective the sooner it was administered after vessel occlusion (Figure 5C; χ²=50.3, df=4, P<0.003, Figure 5D; χ²=25.2, df=2, P<0.003), and in studies that used both male and female animals (χ²=16.3, df=2, P<0.003, Figure 5E; χ²=38.5, df=2, P<0.003, Figure 5F). For infarct volume, efficacy was highest in studies using acepromazine and ketamine combination anesthesia (χ²=29.4, df=7, P<0.003; Figure 5G). For neurobehavioral score, it was highest in those using pentobarbital (χ²=25.5, df=5, P<0.002; Figure 5H) and odds of death were highest when the anesthesia used was not specified (not shown).

Figure 5.

Effect on hypertension on (A) infarct volume, (B) neurobehavioral score. The effect time of administration on (C) infarct volume and (D) neurobehavioral score. The effect of the gender of the animal on (E) infarct volume and (F) neurobehavioral score and the effect of anesthetic used on (G) infarct volume and (H) neurobehavioral score. The shaded gray bar represents the 95% confidence limits of the global estimate. The vertical error bars represent the 95% confidence intervals for the individual estimates. The size of each point reflects the log of the number of animals contributing to that comparison. Each stratification accounts for a significant proportion of the heterogeneity observed between studies.

Larger doses of tPA were associated with higher efficacy for infarct volume (χ²=14.5, df=3, P<0.003; Figure 6A) and higher odds of death (χ²=13.1, df=2, P<0.006; Figure 6B).

Figure 6.

Effect on tissue plasminogen activator (tPA) dose on (A) infarct volume and (B) odds of mortality. The shaded gray bar represents the 95% confidence limits of the global estimate. The vertical error bars represent the 95% confidence intervals for the individual estimates. The size of each point reflects the log of the number of animals contributing to that comparison. Each stratification accounts for a significant proportion of the heterogeneity observed between studies.

For infarct volume, efficacy was higher in studies that used monkeys, where mechanical ventilation was used, that are published in abstract and that occlude the vessel with an autologous clot (not shown).

Discussion

Comparability of Effects of Tissue Plasminogen Activator in Animals and Humans

Here, we provide a detailed systematic analysis of the animal data to support the efficacy of tPA in stroke, and this allows comparison with data from clinical trials in humans. Broadly speaking, there is concordance between the two sets of data. This is reassuring and provides evidence to support the notion that animal models can provide results that are relevant to humans. A very notable feature of the animal data is the marked heterogeneity between studies, implying that the overall estimate of efficacy should be interpreted with some caution. Narrative reviews selectively citing reports at either end of the spectrum of reported efficacy might lead to very different conclusions being drawn about the effect of tPA. This variation justifies the use of systematic reviews in assessing the animal reviews to reduce selection bias and increase precision. The large number of publications also allows the identification of publication bias, thereby allowing readers to make allowances for this in assessing the evidence supporting the efficacy of tPA.

Factors Modifying the Effect of Tissue Plasminogen Activator in Experimental Models

Results from stratified meta-analysis of animal data are remarkably consistent with what is known of the human clinical efficacy of tPA. In both, efficacy is highest where drug is administered within 90 minutes of stroke onset, falls only slightly with delays to 3 hours, and appears to fall away more rapidly thereafter (Wardlaw et al, 2009). In animals, the risk of hemorrhage increases substantially with delays to treatment of longer than ∼4 hours, consistent with data from clinical trials (Hacke et al, 2004, 2008). Regression analysis identified that studies that do not report the use of a sample size calculation were associated with over a 20% overstatement in efficacy. This supports the argument that future studies should use a formal sample size calculation to avoid overestimation of effect size (Macleod et al, 2009).

These data suggest that tPA is without efficacy in hypertensive rats. Given the high prevalence of hypertension in patients with stroke, this might have important implications for its clinical use, and indeed the pivotal trials of tPA excluded patients with severe hypertension. Importantly, spontaneously hypertensive rats differ from other rat strains in factors other than blood pressure; dilatation of arterioles in response to nitric oxide is blunted in spontaneously hypertensive rats, which might impair the development of anastomotic perfusion after middle cerebral artery occlusion (Koller and Huang, 1994); and cortical neurons cultured from the stroke prone spontaneously hypertensive rat are more sensitive to the neurotoxic effects of NMDA (Tagami et al, 1999). More data are required, both animal data from animals with pharmacologically induced hypertension and clinical data on the use of tPA in patients with hypertension.

In these data, the odds of mortality were higher in studies that were considered of poorer quality; that is in those studies that did not report the masked assessment of outcome or randomization; and in studies that did not report the anesthetic used. Few papers reported the number of animals killed prematurely and were therefore not included in the infarct volume or neurobehavioral score data. Although we were not able to assess attrition bias it is an important concern.

The cumulative meta-analysis suggests that estimate of efficacy was stable after data from some 1500 animals had been reported by 2001. Therefore, as for the cumulative MA of thrombolysis in coronary heart disease (Lau et al, 1992), MA in animal experimentation can be used as an important tool to reduce unnecessary animal usage. Most latter studies used tPA as a positive control; in studies testing the efficacy of candidate neuroprotective drugs together with thrombolysis; or in studies exploring the potential for other drugs to modify the risks of tPA treatment.

Implications for Design of Drug Development Studies in Animals and Humans

As noted above, there is reasonable concordance between the results of animal and clinical studies of tPA. There are at least two factors that distinguish the animal data for tPA from that for other drugs where such concordance has not been seen. First, there is a substantial volume of animal data for tPA, derived from 450 experiments and involving a total of 5262 experimental animals. This meta-analysis should therefore give a more precise estimate of the efficacy of tPA less susceptible to the play of chance than is the case for other drugs where there is less animal data.

Second, the interval between the onset of ischemia and the initiation of treatment were similar: uniquely for candidate stroke drugs, tPA has been tested in clinical trials within a time-window within which it also had efficacy in animal studies. Greater concordance of experimental design between animal experimentation and human clinical trials seems likely to be a critical factor in improving translational research.

Methodological Limitations

Although our analyses of the factors influencing true effect of tPA were prespecified and a stringent significance level was chosen to allow for multiple testing, some of the apparent effects may be due to the play of chance. The conclusions above should therefore be viewed as hypothesis generating rather than confirmatory. This meta-analysis has other weaknesses. First, although we consider that our search strategy is likely to have ascertained the majority of relevant publications, it has yet to be validated. We have only been able to include data that has been published in some form: had we been able to include unpublished data, our effect estimates might well have been different as suggested by our adjustment for publication bias. Furthermore, neurobehavioral scales are ordinal, when a large number of data are aggregated together (as here), parametric manipulations have some validity (Lord, 1953). This approach has been used previously by many groups conducting meta-analysis of both clinical trial and animal data but is a limitation of the methodology.

Summary

We examined the biologic factors that influence the efficacy and safety of tPA in thrombotic stroke models. This analysis provides empirical evidence that given a sufficiently large body of experimental and clinical data, animal models and clinical trials can yield broadly similar results. However, these analyses confirm that improvements in experimental design have the potential to make the process of translation from bench to bedside more efficient.

Peter Sandercock is co-chief investigator of the Third International Stroke Trial (IST-3) testing i.v. rt-PA in acute ischemic stroke. IST-3 is an independent, investigator-led trial. In the initial double-blind phase, drug and placebo for the first 300 patients were supplied by Boehringer Ingelheim. The University of Edinburgh and the Lothian Health Board act as joint sponsors. The study is designed, conducted, analyzed, and reported independently of the sponsors and funding agencies. The opinions and conclusions expressed here are those of the authors and do not necessarily reflect those of the UK National Health Service or the Department of Health.