Response to Critique of SAMMPRIS Trial by Abou-Chebl and Steinmetz

Introduction

Even before the early results of SAMMPRIS were published ¹, commentaries that were critical of the trial or that attempted to explain the poor outcome of patients treated with stenting in the trial were published or being submitted for publication ^2–4. Subsequent to the publication of the SAMMPRIS results, other commentaries soon followed ^5–12. As the lead investigators of SAMMPRIS, we welcome the scientific debate on possible limitations of the trial and the implications of the results on management of patients with intracranial stenosis and the design of future trials of this high-risk disease. However, these opinions should be based on an accurate representation of the published data from the trial.

The recent critique of SAMMPRIS in Stroke by Abou Chebl and Steinmetz ¹² fails in this regard. Their critique contains inaccurate references to the study data and protocol, incorrect derivations of event rates in the trial, and selective use of unpublished SAMMPRIS data presented at a scientific meeting. These unpublished data were used to support the authors’ viewpoint while other data from the same presentation that contradicted that viewpoint were omitted. The purpose of this paper is to correct the deficiencies and errors in the critique of SAMMPRIS by Abou-Chebl and Steinmetz.

Incorrect References

In their critique, Abou-Chebl and Steinmetz claim that relationships between specific lesion and procedural variables and peri-procedural complications were misrepresented in the SAMMPRIS primary paper in the New England Journal of Medicine (NEJM). They write “Although in the publication of the SAMMPRIS trial the authors write that vessel size was not related to the risk of complications, there was a greater risk of ICH (10% versus 0%) if the prestent lesion diameter was < 0.6 mm versus > 0.6 mm (p=0.0006); the mean lesion diameter was 0.3 mm in those with ICH versus 0.6 mm without ICH (p<0.0001). It is unclear why vessel size did not emerge as a risk factor in SAMMPRIS but the publication did not state if the analysis was performed in a continuous or dichotomous method, which could have a significant impact on the probability of finding a correlation given the small number of events” ¹².

The truth is that the relationships between lesion and procedure variables and complications in the stenting arm were not addressed in the primary manuscript. Nowhere in the NEJM paper do we write that “vessel size was not related to the risk of complications”. In fact, our group only began the initial analyses of the risk factors for peri-procedural stroke in SAMMPRIS after the NEJM paper had already been published. The data on the risk of ICH in smaller arteries that Abou-Chebl and Steinmetz quote are from an invited presentation at the Society of Vascular Interventional Neurology (SVIN) meeting by one of the SAMMPRIS interventional principal investigators (DF) several weeks after the primary paper was published.

The authors also wrote “The SAMMPRIS authors reported that there was no correlation between balloon size or the ratio of balloon to the stent and the risk of ICH” ¹² and referenced the NEJM paper. This is also incorrect since, as already indicated, we presented no data on factors associated with risk of stroke after stenting in the NEJM paper.

Inaccuracies About the SAMMPRIS Protocol

In trying to make a case that “delayed enrollment may have created a selection bias with artificially lower event rates in the medical arm” ¹², Abou-Chebl and Steinmetz attribute “delayed enrollment” to the fact that “patients had to undergo cerebral angiography with a centralized review before enrollment” ¹². However, as described in the SAMMPRIS design paper ¹³, the central review was only required in patients with a local angiogram reading of 70–79% stenosis to ensure that we did not have a high number of patients enrolled with a local reading of 70–79% and central reading of < 70%. This policy was implemented by requiring that JPEG images of angiograms read locally as 70% – 79% stenosis be sent by e-mail to both interventional principal investigators (CD and DF), one of whom typically responded directly back to the site within an hour or so of receiving the email, and always on the same day. Hence the JPEG requirement, which was used in 143 (32%) of the 451 patients enrolled in the trial, did not delay randomization, as suggested by the authors of the critique.

Even if the premise that the medical event rate was “artificially lower” ¹² because of selection bias had some validity (which there is no evidence for), randomization would have ensured that the stenting arm was also at an “artificially lower” risk. The authors provide no explanation of why the stenting complication rate was still so high in this “artificially lower” risk population, and why they would not expect it to be even higher in a higher risk population. Moreover, the contention of Abou-Chebl and Steinmetz that patients treated soon after their qualifying event are more likely to benefit from stenting ¹² is not supported by the SAMMPRIS data. In an article on the peri-procedural complications in SAMMPRIS in this issue of Stroke, Fiorella et al.¹⁴ provide data showing that the 30-day absolute risk reduction from medical therapy in patients enrolled within 7 days after their qualifying event was 8.7%, which is very similar to the 9.3% absolute risk reduction from medical therapy in patients enrolled beyond 7 days after their qualifying event.

The authors also incorrectly state that another effect of the central review of angiograms was an “increase in the number of procedures” ¹². In fact, central review of angiograms had no effect other than to exclude ineligible patients.

Incorrect Calculations of New Event Rates in SAMMPRIS

In trying to make a case for “inadequacy of operator experience in SAMMPRIS” ¹², Abou-Chebl and Steinmetz calculate that the angiographic stroke rate was high at 1.3%, and state that the number of patients who received two stents was 9% ¹². Both of these calculations are incorrect.

We are not sure how the authors calculated the 1.3% angiographic stroke rate, but suspect that it was based on the NEJM paper ¹, in which we reported that three of the 224 patients in the stenting arm whose stenting procedure was aborted had a stroke (3/224 = 1.3%). This calculation makes incorrect assumptions about both the numerator and denominator in the calculation of the angiographic stroke rate. For the numerator, an assumption is made that the three strokes were all attributed to angiography when in fact only one was. As discussed in the accompanying article by Fiorella et al ¹⁴, two patients had strokes 3 and 4 days after angiograms that were done as part of the planned stenting procedure. These angiograms showed that the previously stenotic artery had progressed to occlusion so stenting was aborted in both cases. Thus, the strokes in these two patients, which were both in the territory of the occluded artery, were almost certainly caused by the natural history of the intracranial occlusion and not a complication of angiography performed several days earlier.

The denominator used in the calculation is also incorrect since it does not take into account the 451 diagnostic angiograms done in all the patients who were enrolled in the trial. In fact, it is not possible to calculate the true angiographic complication rate in all patients considered for the trial since we do not know the total number of patients who underwent angiography as potential candidates for SAMMPRIS, nor do we know who, if any, of this entire group had an angiographic stroke. The most reliable estimate of the angiographic stroke rate in SAMMPRIS can be obtained from the patients who underwent a single vessel angiogram paid for by the study to confirm an intracranial stenosis detected on CTA or MRA since we documented any adverse events in those patients. Of the 149 patients who underwent a study paid angiogram, none (0%) had an angiographic stroke.

Based on the SAMMPRIS presentation at SVIN, Abou-Chebl and Steimetz state that 9% of patients in SAMMPRIS “received 2 stents”, which is another indicator of “the inadequacy of operator experience in SAMMPRIS” ¹². However, Abou-Chebl and Steinmetz have misinterpreted the data presented at SVIN indicating that 91% of patients were successfully treated with a single stent. In the other 9%, either no devices were opened (stenting was not attempted in seven of these cases), or more than one system was introduced due to an inability to deliver or deploy the original device. Thus, the statement that 9% of stented patients in SAMMPRIS “received 2 stents” is incorrect. In fact, the technical outcomes of balloon angioplasty and stenting in SAMMPRIS were excellent, with success rates that are comparable to the technical success rates achieved in the US Wingspan and NIH Wingspan registries ^15,16. Details of the technical outcomes in SAMMPRIS are provided in the paper by Fiorella at al ¹⁴ in this issue of Stroke.

Abou-Chebl and Steinmetz also recalculate what the 30-day event rate would have been in the stenting arm had patients presenting with perforator strokes been excluded from SAMMPRIS. The rationale for doing this is based on their premise that stenting is unlikely to benefit patients presenting with perforator strokes and that these patients are at higher risk of peri-procedure perforator stroke ¹². With the knowledge that the most common peri-procedural strokes in SAMMPRIS were perforator strokes (from attendance at the SVIN presentation), they calculate that “exclusion of these patients from SAMMPRIS” (presumably all patients whose qualifying event was a perforator stroke) could have reduced the stroke and death rate in the stenting arm at 30 days from 14.7% to 9.4%, and could have “prevented half of the ischemic strokes” ¹².

This analysis also makes several incorrect assumptions in calculating the 9.4% rate, including the fact that if patients presenting with perforator infarcts were excluded (45 patients in the stenting arm had an acute infarct in a perforator territory on baseline imaging ¹⁴), the denominator would be substantially lower than all 224 patients in the stenting arm they used in the calculation, which would of course make the stroke rate higher. More importantly, the authors assume that the patients who suffered a perforator stroke during stenting in SAMMPRIS were those patients who had qualified for the study with a perforator infarct. In fact, as described in the article in this issue of Stroke by Fiorella et al ¹⁴, none of 45 patients who presented with an acute perforator infarct on baseline imaging in SAMMPRIS had a peri-procedural perforator stroke after stenting. So the 30-day primary endpoint rate of 9.4% derived by Abou-Chebl and Steinmetz in patients without a perforator presentation is baseless. In fact, the 30-day primary event rate in patients who did not present with a perforator stroke is actually higher than the 14.7% rate in the entire stenting cohort in SAMMPRIS because patients presenting with perforator strokes had a lower rate of primary endpoints compared with patients presenting with TIA or non-perforator strokes (see rates of ischemic and hemorrhagic endpoints relative to presentation in accompanying article by Fiorella et al. ¹⁴).

The findings from WASID that patients who present with a lacunar stroke have the same risk of recurrent stroke as patients presenting with non-lacunar stroke, and that recurrent stroke in the territory of the stenotic artery in patients presenting with a lacunar stroke is typically distal to the stenosis justify including those patients in SAMMPRIS ¹⁷. Additionally, the results of SAMMPRIS show unequivocally that including those patients was not a contributing factor to the high peri-procedural stroke rate in the trial.

Selective Presentation and Omission of Unpublished SAMMPRIS Data

In trying to build on their case for “inadequacy of operator experience in SAMMPRIS” ¹², Abou-Chebl and Steinmetz provide SAMMPRIS data presented at SVIN showing that the rate of any peri-procedural hemorrhagic stroke was higher at lower enrolling sites in SAMMPRIS compared with higher enrolling sites ¹². However, the rate of peri-procedural ischemic stroke in SAMMPRIS was lower at the lower enrolling sites, which resulted in overall 30-day rates of any stroke (i.e., ischemic or hemorrhagic) among patients who underwent stenting of 13.5% at sites that enrolled ≥ 12 patients (12 being the median) and 14.7% at sites that enrolled < 12 patients (p = 0.77) ¹. In their critique, the authors indicate that the lack of association between operator experience and risk of peri-procedural stroke was unexpected but that “the important question is what was the risk of complications based on previous operator experience using the Wingspan system in the treatment of atherosclerotic disease? Then and only then might the real association between experience and complications be found” ¹². What the authors fail to mention in their critique, however, is that the data addressing this question were provided in the same presentation at the SVIN meeting from which the hemorrhagic stroke data above were selectively extracted.

The paper by Derdeyn et al. ¹⁸ in this issue of Stroke on the relationships between operator and site experience with Wingspan and 30-day peri-procedural stroke rates in SAMMPRIS provides the data from the SVIN presentation omitted by Abou-Chebl and Steinmetz in their critique, as well as other related analyses. In brief, these analyses show that interventionists credentialed for SAMMPRIS with lower numbers of Wingspan stents did not have higher rates of peri-procedural stroke than interventionists credentialed with higher numbers of Wingspan stents ¹⁷. Additionally, the rates of peri-procedural stroke in SAMMPRIS were similar amongst interventionists that contributed the most patients to the two previous Wingspan registries compared with other interventionists in SAMMPRIS ^15–17.

Conclusion

The results of SAMMPRIS were surprising and disappointing to many investigators in our field, some of whom are looking for alternative explanations for the study results ^{2,5, 12}. However, the most obvious and logical explanation for the current SAMMPRIS data remains the one that led to enrollment in the trial being stopped: aggressive medical therapy is far superior to stenting for preventing stroke in these high-risk patients. While we encourage debate on the implications and limitations of SAMMPRIS to continue, it is important that this debate is based on knowledge of the SAMMPRIS protocol and accurate representation of data from the trial. The critique by Abou-Chebl and Steinmetz ¹² failed on multiple levels in this regards, as described above.

One strategy that could be taken to avoid publication of incorrect study data by authors who did not participate in the original research is for journals to re-evaluate the policy that allows authors to quote unpublished data as long as the source of that data is referenced correctly and that also allows derivation of new event rates without access to the full data set. Simply quoting the source of unpublished data does not provide readers with an opportunity to review the data and verify that the authors’ representation is accurate and complete. Prohibiting authors from quoting unpublished data will also serve to protect the right of the original investigators to publish and interpret their data first before other authors have the opportunity to report and potentially misinterpret that data.