Recommendations for studying the safety, efficacy and durability of intracranial aneurysm devices

Approximately 3% of adults harbour an unruptured intracranial aneurysm.¹ Rupture causes aneurysmal subarachnoid haemorrhage, a stroke with a 3-month case-fatality of 40%. Aneurysms, unruptured or ruptured, can be treated to prevent future bleeding. However, the overarching goal of treatment is not to decrease bleeding risk, but to increase quality-adjusted-life-years.

Microsurgical clipping was the standard for intracranial aneurysm treatment for decades until the International subarachnoid aneurysm trial (ISAT) randomized trial showed that endovascular coiling of ruptured aneurysms results in better patient outcomes than clipping where both treatments were suitable.² Although most patients were treated outside the trial and the majority of included patients had a good clinical condition on admission, the ISAT results have been generalized to almost all patients with aneurysmal subarachnoid haemorrhage, as well as to patients with unruptured intracranial aneurysms, although this patient population was not studied in ISAT. In the years after completion of ISAT, new endovascular treatment options for intracranial aneurysms were introduced. Though developed for large or wide-necked intracranial aneurysms, these techniques are now also applied to small aneurysms amenable for regular coiling or clipping.

A systematic review of 114 studies involving 106,433 patients with 108,263 unruptured intracranial aneurysms found higher complication rates with advanced endovascular techniques compared to regular coiling.³ Without randomised controlled trials (RCTs), it remains unclear whether these advanced techniques are safer or more effective than coiling, clipping or observation. A broader systematic review of 1356 studies covering 410,993 patients, showed that studies claiming safety, effectiveness or durability of intracranial aneurysm treatment often had methodological flaws and incomplete reporting of relevant outcomes supporting these claims.⁴ Common methodological concerns included single-arm designs, short follow-up and inconsistent outcome definitions. Only 2% of studies were prospective comparisons.

The interaction between industry, academics, regulators, health insurance companies and patient representatives is complex. The Food and Drug Administration (FDA) and European Medicines Agency (EMA) regulate device approval, classifying clips and coils as class II devices in the United States and all other devices as class III (high risk). In Europe all devices are classified as high risk. Neither the FDA nor EMA mandate comparative studies, even when a treatment for the same indication already exists. The result is that devices may be approved without even demonstrating non-inferiority to current best practice. Registry-based studies, often used in lieu of RCTs, carry substantial bias due to lack of control groups, selection of low-risk patients and self-assessment by financially invested clinicians.⁵

If no alternative treatment exists for a specific indication, observational studies without control groups may suffice for investigating safety and effectiveness. However, when other treatments for a specific indication are available, new devices should undergo rigorous evaluation. We therefore propose the following framework:

1. RCT requirement: New devices must be tested against current treatment options by performing RCTs when alternative treatment options for a specific indication exist.

2. Clear indications: Indications should specify aneurysm features (size, neck width, location, rupture status) and patient characteristics (age, sex and clinical condition on admission). Subsequent market approval should be based on these characteristics.

3. Phase 2 trials: Phase 2 RCTs should focus on safety and efficacy, starting with unruptured intracranial aneurysms to avoid confounding from complications related to aneurysmal subarachnoid haemorrhage.

4. Phase 3 trials: After phase 2 RCTs shows that the new device is superior or equally safe and efficacious to current treatment options, approval can be granted with the requirement to perform a multicentre phase 3 RCT assessing patient outcomes and cost-effectiveness within a specific healthcare system.

5. Registry inclusion: Eligible but non-enrolled patients who are treated outside the trial should be tracked in a registry to contextualize the generalizability of trial data.

6. Durability: Long-term durability (eg, prevention of rupture) should be studied in post-marketing registries, in which every participant is prospectively registered before the treatment occurs.

7. Standardized definitions: To facilitate comparisons between studies and aggregate results in individual patient data meta-analyses, standardised variable coding, definitions of safety, efficacy and durability are needed. A consensus on such definitions is necessary, involving stakeholders from industry, academics, regulators and patient representatives.

8. Standardized timepoints: Safety should be assessed at 30 days (phase 2), efficacy at 6 and 18 months (phase 3) and durability at 5 years (post-marketing).

9. Independent assessments: Imaging and clinical outcomes should be evaluated by independent reviewers or labs.

10. Protocol registration: Study protocols, including definitions, endpoints, statistical analysis plans and time-points for outcome assessments, must be preregistered and time-stamped before data-analysis.

11. No conflicts of interest: RCTs should be designed and conducted by unbiased investigators and institutions.

Adopting these recommendations will improve device evaluation, enhance the quality of evidence guiding intracranial aneurysm treatment and support sound clinical decision-making.

Author contributions

V.V., M.D.I.V.: study supervision, drafting the first manuscript, approving the final manuscript. T.R.M., T.K., P.T., G.S.C., G.J.E.R.: editing the manuscript.

Conflicts of interest

No conflicting interests, financial or otherwise, are present.

Funding

M.D.I.V. received funding from Bayer for investigator-initiated research unrelated to the current study.

Data availability

Only published data was used and referenced.

Guarantor

Dr. Victor Volovici.