New York State: Comparison of Treatment Outcomes for Unruptured Cerebral Aneurysms Using an Instrumental Variable Analysis

Abstract

Background

There is wide regional variation in the predominant treatment for unruptured cerebral aneurysms. We investigated the association of elective surgical clipping and endovascular coiling with mortality, readmission rate, length of stay, and discharge to rehabilitation.

Methods and Results

We performed a cohort study involving patients with unruptured cerebral aneurysms, who underwent surgical clipping or endovascular coiling from 2009 to 2013 and were registered in the Statewide Planning and Research Cooperative System database. An instrumental variable analysis was used to investigate the association of treatment technique with outcomes. Of the 4643 patients undergoing treatment, 3190 (68.7%) underwent coiling, and 1453 (31.3%) underwent clipping. Using an instrumental variable analysis, we did not identify a difference in inpatient mortality (marginal effect, 0.13; 95% CI, −0.30, 0.57), or the rate of 30-day readmission (marginal effect, −1.84; 95% CI −4.06, −0.37) between the 2 treatment techniques for patients with unruptured cerebral aneurysms. Clipping was associated with a higher rate of discharge to rehabilitation (marginal effect, 2.31; 95% CI 0.21, 4.41), and longer length of stay (β, 2.01; 95% CI 0.85, 3.04). In sensitivity analysis, mixed-effect regression, and propensity score, adjusted regression models demonstrated identical results.

Conclusions

Using a comprehensive all-payer cohort of patients in New York State with unruptured cerebral aneurysms, we did not identify an association of treatment method with mortality or 30-day readmission. Clipping was associated with a higher rate of discharge to rehabilitation and longer length of stay.

Unruptured cerebral aneurysms affect 1% to 3% of the population.^1–4 They are increasingly diagnosed in asymptomatic individuals, as a result of the widespread use of noninvasive imaging techniques for unrelated indications.¹ Rupture of these lesions can have catastrophic consequences, with major morbidity and mortality.^1,5,6 When a decision is made for intervention, most aneurysms are amenable to both surgical clipping and endovascular coiling.¹ However, the minimally invasive nature of the endovascular approaches has resulted in a dramatic decrease in the number of patients undergoing surgical clipping.¹ In addition, the volume of unruptured aneurysms treated with any form of intervention has increased in recent years, almost exclusively driven by the widespread availability and use of coiling.^5,6 These trends have fueled an ongoing debate about the relative effectiveness of the 2 treatments available for patients with unruptured cerebral aneurysms.^5,6

However, no randomized trials have been performed to answer this question in asymptomatic patients.¹ Observational studies have demonstrated mixed results.^7–21 The main limitation of such investigations is not accounting for unmeasured confounding. Patients included in prior retrospective studies have been selected for either procedure in advance. This selection reflects the different preferences and backgrounds of the treating physicians, as well as specific patient characteristics, and anatomic information such as aneurysm size and location. Administrative databases lack this granularity, limiting the ability to control for such confounders. There has been no prior study attempting to account for these limitations through different analytic approaches in an adult cohort of all ages.

We used the New York Statewide Planning and Research Cooperative System (SPARCS)²² to study the association of treatment technique with mortality, discharge to rehabilitation, 30-day readmission, and length of stay (LOS) for patients undergoing surgical clipping or endovascular coiling for unruptured cerebral aneurysms. An instrumental variable analysis was used in an attempt to control for unmeasured confounding and simulate the effect of randomization.

Methods

New York Statewide Planning and Research Cooperative System

Institutional Review Board approval was obtained for this study. All patients with unruptured cerebral aneurysms who were registered in the SPARCS (New York State Department of Health, Albany, NY)²² database between 2009 and 2013 were included in the analysis. For these years, SPARCS contains patient-level details for every hospital discharge, ambulatory surgery, and emergency department admission in New York State as coded from admission and billing records. More information about SPARCS is available at https://www.health.ny.gov/statistics/sparcs/. This work was based on a database with de-identified data. No informed consent was required for the execution of this work.

Cohort Definition

In order to establish the cohort of patients, we used International Classification of Disease-9-Clinical Modification codes to identify patients in the registry who underwent surgical clipping (International Classification of Disease-9-Clinical Modification procedure code 39.51) or endovascular coiling (1 of the following procedure International Classification of Disease-9-Clinical Modification codes: 39.52, 39.72, 39.75, or 39.76) for unruptured (diagnosis International Classification of Disease-9-Clinical Modification code 437.3, in any position) cerebral aneurysms between 2009 and 2013. These are the only administrative codes available for the diagnosis and treatment of cerebral aneurysms and have been used consistently in all prior observational literature on the subject.^1,5,8,11,19

Outcome Variables

The primary outcome variable was mortality during the initial hospitalization after treatment of an unruptured cerebral aneurysm. Secondary outcomes were LOS during the initial hospitalization, the rate of discharge to rehabilitation facility (any facility other than the patient’s home), and 30-day postdischarge readmission to any hospital.

Exposure Variables

The primary exposure variable was the treatment method (surgical clipping versus endovascular coiling).

Covariates (Table S1) used for risk adjustment were age, gender, race (African–American, Hispanic, Asian, white, other), and insurance (private, Medicare, Medicaid, uninsured, other). The comorbidities used for risk adjustment were diabetes mellitus, smoking, chronic lung disease, hypertension, hypercholesterolemia, peripheral vascular disease, congestive heart failure, coronary artery disease, history of stroke, transient ischemic attack, alcohol abuse, obesity, chronic renal failure, and coagulopathy. Only variables that were defined as “present on admission” were considered part of the patient’s preadmission comorbidity profile.

Hospital and physician average, annual case (clipping and coiling) volume were also used as risk adjusters in all our regression models.

Statistical Analysis

The association of treatment technique with our outcome measures was examined in a multivariable setting. Patients undergoing surgical clipping or endovascular coiling in our cohort were nonrandomly selected for either procedure based on provider and patient characteristics. Attempting to account for this unmeasured confounding, and to simulate the effect of randomization, we used an instrumental variable analysis, an econometric technique.²³ The regional ratio of coiling (county-level coiling ratio) was used as an instrument for the treatment received. The county coiling ratio was calculated by dividing the average number of coiling procedures in a county by the total number of procedures (clipping and coiling) in the same county. The regional rate of a procedure has been used before to create pseudorandomization on the treatment method, using an instrumental variable analysis.²⁴ In sensitivity analysis, we used the differential distance of the patient’s residence to facilities preferentially offering coiling versus clipping. Although the results were qualitatively the same, this second instrumental variable approach had minimal ability to discriminate between treatments, and resulted in high variance. Therefore, this was not used further.

A good instrument is not associated with the outcome other than through the exposure variable of interest (a requirement known as the exclusion restriction criterion).²⁴ In our case it is unlikely that the regional ratios of coiling would be associated with treatment mortality in any way other than the choice of treatment. A 2-stage least-squares method was used for the calculation of the coefficients. The value of the F statistic in the first stage of the 2-stage least-squares approach was 25, which is consistent with a strong instrument (F statistic >10), based on a practical rule.²³

A probit regression was used for the categorical outcomes (mortality, discharge to rehabilitation, 30-day readmission),²⁵ and a linear regression for the linear outcomes (LOS). We used the probit regression for the instrumental variable analysis, because the probit function is used almost exclusively in the literature for instrumental variable analysis with binary outcomes. Since the coefficients produced by the probit function are not interpretable, we used the marginal effects of our independent variables instead. The marginal effects are the partial derivatives of the coefficients, and reflect the change in the probability of the dependent variable, for 1 unit change in the independent variable, at the average value of all other covariates.

In order to demonstrate the robustness of our data in a sensitivity analysis, we used standard techniques to account for measured confounding while accounting for clustering at the hospital level. For categorical outcomes, we used a logistic regression model with hospital ID as a random effects variable. A logistic regression model was used in order to report clinically interpretable odds ratios. The covariates used for risk adjustment in these models were as follows: age, gender, race, insurance, hospital ID, average annual provider coiling volume, average annual provider clipping volume, average annual hospital coiling volume average annual hospital clipping volume, and all the comorbidities mentioned previously. Due to the small number of events associated with mortality, we did not control for all these covariates, in order to avoid unstable models. Instead we fitted the model used stepwise backwards elimination, and eventually only the 4 most significant variables were included in the final model. In an alternative way to control for confounding, we used a propensity-adjusted (with deciles of propensity score) logistic regression model. We calculated the propensity score of coiling with a separate logistic regression model, using all the covariates mentioned previously. For continuous outcomes, we performed similar analyses using linear models. Logarithmic transformation of the values of LOS yielded identical results and is therefore not reported further.

Regression diagnostics were used for all models. Number needed to treat was calculated when appropriate. All results are based on 2-sided tests, and the level of statistical significance was set at 0.05. This study, based on 4643 patients, has sufficient power (90%) at a 5% type I error rate to detect relative differences in mortality, as small as 10.2%. Statistical analyses were performed using Stata version 13 (StataCorp, College Station, TX), the 64-bit version of R.3.1.0 (R Foundation for Statistical Computing), and SPSS version 22 (IBM, Armonk, NY).

Results

Patient Characteristics

In the selected study period, there were 4643 patients undergoing treatment for unruptured cerebral aneurysms (mean age was 55.0 years, with 76.7% females) who were registered in SPARCS. Of these, 3190 (68.7%) underwent surgical clipping, and 1453 (31.3%) underwent endovascular coiling. The characteristics of the 2 cohorts at baseline can be seen in Table1. Patient characteristics stratified based on county coiling ratio are demonstrated in Table S2.

Table 1.

Patient Characteristics

	Total		Coiling		Clipping		P Value
	N=4643		N=3190		N=1453
	Mean	SD	Mean	SD	Mean	SD
Age, y	55.03	14.12	55.07	15.19	54.92	11.42	0.7371
	N	%	N	%	N	%	P Value
Gender
F	3561	76.7	2471	77.5	1090	75.0	0.072
M	1082	23.3	719	22.5	363	25.0
Diabetes
−	4165	89.7	2863	89.7	1302	89.6	0.876
+	478	10.3	327	10.3	151	10.4
Smoking
−	3744	80.6	2627	82.4	1117	76.9	<0.001
+	899	19.4	563	17.6	336	23.1
Obesity
−	4355	93.8	3002	94.1	1353	93.1	0.212
+	288	6.2	188	5.9	100	6.9
Transient ischemic attack
−	4633	99.8	3185	99.8	1448	99.7	0.303
+	10	0.2	5	0.2	5	0.3
Ischemic stroke
−	4622	99.5	3175	99.5	1447	99.6	0.817
+	21	0.5	15	0.5	6	0.4
Coronary artery disease
−	4300	92.6	2949	92.4	1351	93.0	0.545
+	343	7.4	241	7.6	102	7.0
Chronic lung disease
−	3849	82.9	2660	83.4	1189	81.8	0.193
+	794	17.1	530	16.6	264	18.2
Congestive heart failure
−	4550	98.0	3126	98.0	1424	98.0	0.910
+	93	2.0	64	2.0	29	2.0
Coagulopathy
−	4606	99.2	3168	99.3	1438	99.0	0.218
+	37	0.8	22	0.7	15	1.0
Chronic renal failure
−	4625	99.6	3179	99.7	1446	99.5	0.458
+	18	0.4	11	0.3	7	0.5
Hypertension
−	2196	47.3	1558	48.8	638	43.9	0.002
+	2447	52.7	1632	51.2	815	56.1
Hypercholesterolemia
−	3316	71.4	2306	72.3	1010	69.5	0.054
+	1327	28.6	884	27.7	443	30.5
Alcohol
−	4554	98.1	3138	98.4	1416	97.5	0.038
+	89	1.9	52	1.6	37	2.5
Peripheral vascular disease
−	4513	97.2	3099	97.1	1414	97.3	0.775
+	130	2.8	91	2.9	39	2.7

Inpatient Mortality

Overall, 9 (0.62%) inpatient deaths were recorded after clipping and 15 (0.47%) after coiling (Table2). Clipping was not associated with increased mortality in comparison to coiling (odds ratio, 1.28; 95% CI, 0.56, 2.94) in unadjusted analysis. Likewise, there was no association of treatment technique with mortality (marginal effect, 0.13; 95% CI, −0.30, 0.57) after using a probit regression with instrumental variable analysis (Table3). This persisted in a mixed-effects logistic regression model (odds ratio, 1.94; 95% CI, 0.67, 5.58) and a propensity score adjusted model (odds ratio, 1.60; 95% CI 0.63, 3.94).

Table 2.

Outcomes

	Total	Coiling	Clipping	P Value
Death	24 (0.52%)	15 (0.47%)	9 (0.62%)	0.511
Discharge to rehabilitation	759 (16.35%)	312 (9.78%)	447 (30.76%)	<0.0001
30-day readmission	309 (6.65%)	202 (6.33%)	107 (7.36%)	0.191
Length of stay	2 (3)	2 (2)	4 (4)	<0.0001

The numbers displayed represent N (%), except from length of stay where mean (standard deviation) are displayed.

Table 3.

Multivariable Models Examining the Association of Surgical Clipping With Outcomes

	Inpatient Mortality		Discharge to Rehabilitation		30-Day Readmission		Length-of-Stay*
	ME (95% CI)	P Value	ME (95% CI)	P Value	ME (95% CI)	P Value	β (95% CI)	P Value
Instrumental variable analysis†	0.13 (−0.30, 0.57)	0.544	2.31 (0.21, 4.41)	<0.001	−1.84 (−4.06, 0.37)	0.103	2.01 (0.85, 3.04)	<0.001
	OR (95% CI)	P Value	OR (95% CI)	P Value	OR (95% CI)	P Value	β (95% CI)	P Value
Mixed-effects logistic regression‡	1.94 (0.67, 5.58)	0.219	6.85 (5.54, 8.48)	<0.001	0.80 (0.61, 1.06)	0.125	3.86 (3.29, 4.43)	<0.001
Propensity score adjusted logistic regression§	1.62 (0.63, 3.93)	0.337	6.21 (5.02, 7.69)	<0.001	0.82 (0.63, 1.09)	0.321	3.58 (2.99, 4.18)	<0.001

ME indicates marginal effects; OR, odds ratio.

^*All regressions were based on linear models.

^†County coiling rate was used as an instrument of coiling.

^‡Hospital ID was used as a random effects variable.

^§The propensity score was calculated using the following variables: sex, race, insurance, medical comorbidities.

Discharge to Rehabilitation

Overall, 447 (30.76%) were discharged to rehabilitation after clipping and 312 (9.78%) after coiling (Table2). Clipping was associated with an increased rate of discharge to rehabilitation in comparison to coiling (odds ratio, 4.01; 95% CI, 3.41, 4.71) in the unadjusted analysis. This persisted (marginal effect, 2.31; 95% CI 0.21, 4.41) after using a probit regression with instrumental variable analysis (Table3). We found similar results in a mixed-effects logistic regression model (odds ratio, 6.85; 95% CI, 5.54, 8.48) and a propensity-score adjusted model (odds ratio, 6.21; 95% CI 5.02, 7.69). This corresponded to 5 patients who needed to be treated with coiling to prevent 1 discharge to rehabilitation.

30-Day Readmission

Overall, 107 (7.36%) were readmitted within 30 days after clipping and 202 (6.33%) after coiling (Table2). Clipping was not associated with increased rate of 30-day readmission in comparison to coiling (odds ratio, 1.18; 95% CI, 0.93, 1.51) in the unadjusted analysis. Similarly, there was no association (marginal effect, −1.84; 95% CI −4.06, 0.37) after using a probit regression with instrumental variable analysis (Table3). We found similar results in a mixed-effects logistic regression model (odds ratio, 1.12; 95% CI, 0.90, 1.45) and a propensity-score adjusted model (odds ratio, 1.02; 95% CI, 0.85, 1.32).

Length-of-Stay

The average LOS was 4 days (SD 2) after clipping, and 2 days (SD 2) after coiling (Table2). Clipping was associated with increased LOS in comparison to coiling (β, 3.15; 95% CI, 2.67, 3.62) in the unadjusted analysis. This persisted (β, 2.01; 95% CI 0.85, 3.04) after using a linear regression with instrumental variable analysis (Table3). We found similar results in a mixed-effects linear regression model (β, 3.86; 95% CI, 3.29, 4.43) and a propensity-score adjusted linear regression model (β, 3.58; 95% CI, 2.99, 4.18).

Discussion

Using a comprehensive all-payer cohort of patients in New York State with unruptured aneurysms, we did not identify an association of treatment method with mortality, or 30-day readmission. Clipping was associated with a higher rate of discharge to rehabilitation and longer LOS. Our results were robust when considering several advanced observational techniques to account for measured and unmeasured confounders. Endovascular coiling has seen explosive growth in recent years, and is currently performed by multiple specialties, without strict certification criteria. This is contributing to an ongoing debate about the relative effectiveness in the community of these 2 treatment interventions for unruptured cerebral aneurysms.^5,6

Several observational studies have compared the outcomes of clipping and coiling for this population.^7–21 The majority was based on the National Inpatient Sample, a 20% sample of all discharges from US hospitals.^{7–9,12,13,15,26} Brinjikji et al⁸ demonstrated that coiling was associated with improved inpatient survival. In another investigation, focusing on the elderly, and utilizing a 5% Medicare sample, Qureshi et al¹¹ did not find a difference in the outcomes of the 2 techniques. The main limitation of this analysis is the small sample size, limiting its ability to detect any difference in mortality. Another group,¹⁰ utilizing the MarketScan database, did not identify a survival benefit of either treatment, although clipping was associated with higher rate of unfavorable discharge. The use of a database with voluntary participation restricts the generalizability of their findings to other populations.

These prior analyses have some common methodologic limitations. Multicenter studies are vulnerable to clustering at the hospital level. Previous authors did not evaluate or adjust for this bias. Most importantly, all the analytical methods used accounted, to some degree, for known confounders. Although this may be adequate in some studies, the selection of patients for either treatment prior to the analysis introduces significant unmeasured confounding. Patients may be selected for coiling because of favorable anatomy, aneurysm location, or general health. Physician or patient preference, as well as provider training and specialty, might affect that decision too. Not accounting for this dimension of confounding puts the robustness of their findings into question. Our study, purposefully addressing this potential bias, utilized an econometric technique to attempt to account for unmeasured confounding, and simulate pseudorandomization.

The present analysis demonstrated similar inpatient mortality levels with prior national or local investigations.^7–21 However, it appears that significantly more patients were not discharged home among those undergoing clipping in comparison to coiling. Although disposition does not necessarily reflect functional outcome, it is likely that clipped patients were more impaired postoperatively.^5,6,27 Another possibility is that the increased invasiveness of clipping, resulted in longer LOS (as we also observed in our cohort), and decreased mobility, increasing the need for rehabilitation at discharge. Prospective registries can identify differences between the 2 techniques in terms of functional outcomes, which can be subsequently studied definitively in randomized trials. In this direction, the NeuroPoint Alliance has created the first module for a cerebrovascular registry, with results expected in the near future.²⁸

Our study has several limitations common to administrative databases. Residual confounding could account for some of the observed associations. However, this is minimized to the extent that we are using a strong instrument for coiling, as indicate by our F statistic. A strong instrument should be less susceptible (but not immune) to bias than a weak instrument. In addition, coding inaccuracies will undoubtedly occur and can affect our estimates. The accuracy of procedure codes for cerebral aneurysm treatment remains unknown. In 1 study examining the accuracy of a regional administrative database, International Classification of Disease-9 codes (437.3) for cerebral aneurysm demonstrated a sensitivity of 85%, specificity of 81%, and a positive predictive value of 79%.²⁹ Although SPARCS includes all hospitals from the entire New York State, the generalization of this analysis to the entire US population is uncertain. SPARCS does not provide any clinical information on the structure, size, or location of the aneurysms, which are important factors in cerebrovascular neurosurgery. However, the use of the instrumental variable analysis is attempting to control for unknown confounders such as these.

Additionally, we were lacking posthospitalization and long-term data on our patients. There is a potential long-term advantage of clipping (more definitive and long-term treatment with lower follow-up requirements and lower retreatment rates), which may offset short-term differences between the 2 procedures. Quality metrics (ie, modified Rankin score) are also not available through this database, and therefore we cannot compare the 2 treatment techniques on these outcomes. Finally, causality cannot be definitively established based on observational data, despite the use of advanced techniques, such as the instrumental variable analysis.

Conclusions

Despite the widespread use of coiling in the treatment of unruptured cerebral aneurysms, there is still considerable debate about the relative effectiveness of surgical clipping and endovascular coiling in real-world practice. Using a comprehensive all-payer cohort of patients in New York State with unruptured aneurysms, we did not identify an association of treatment method with mortality, or 30-day readmission. Clipping was associated with a higher rate of discharge to rehabilitation and longer LOS. Our results were robust when considering several advanced observational techniques to account for measured and unmeasured confounders.

Sources of Funding

Supported by grants from the National Institutes of Health (NIH) Common Fund (U01-AG046830) and the National Center for Advancing Translational Sciences (NCATS) of the NIH (Dartmouth Clinical and Translational Science Institute-UL1TR001086). The funders had no role in the design or execution of the study.

Disclosures

None.