The Chiari Severity Index: A Preoperative Grading System for Chiari Malformation Type 1

Abstract

Background

To develop evidence-based treatment guidelines for Chiari Malformation Type 1 (CM-1), preoperative prognostic indices capable of stratifying patients for comparative trials are needed.

Objective

To develop a preoperative Chiari Severity Index (CSI) integrating the clinical and neuroimaging features most predictive of long-term patient-defined improvement in quality of life (QOL) after CM-1 surgery.

Methods

We recorded preoperative clinical (e.g. headaches, myelopathic symptoms) and neuroimaging (e.g. syrinx size, tonsillar descent) characteristics. Brief follow-up surveys were administered to assess overall patient-defined improvement in QOL. We used sequential sequestration to develop clinical and neuroimaging grading systems and conjunctive consolidation to integrate these indices to form the CSI. We evaluated statistical significance using the Cochran-Armitage test and discrimination using the c-statistic.

Results

Our sample included 158 patients. Sequential sequestration identified headache characteristics and myelopathic symptoms as the most impactful clinical parameters, producing a clinical grading system with improvement rates ranging from 81% (grade 1) to 58% (grade 3) (p=0.01). Based on sequential sequestration, the neuroimaging grading system included only the presence (55% improvement) or absence (74% improvement) of a syrinx ≥ 6 mm (p=0.049). Integrating the clinical and neuroimaging indices, improvement rates for the CSI ranged from 83% (grade 1) to 45% (grade 3) (p=0.002). The combined CSI had moderately better discrimination (c=0.66) than the clinical (c=0.62) or neuroimaging (c=0.58) systems alone.

Conclusion

Integrating clinical and neuroimaging characteristics, the CSI is a novel tool that predicts patient-defined improvement following CM-1 surgery. The CSI may aid preoperative counseling and stratify patients in comparative effectiveness trials.

Introduction

Diagnosed in approximately 1% to 4% of patients undergoing brain or cervical spine magnetic resonance imaging (MRI) studies,^{1, 2} Chiari Malformation Type 1 (CM-1) is a common and often debilitating neurosurgical condition. In recent years, there has been a growing emphasis on outcomes research, including efforts to compare the results of various surgical approaches.^{3, 4} To help facilitate more objective comparisons, several groups have produced and/or validated different metrics to evaluate treatment outcomes.^5–8 However, despite this growing interest in improved research methodology, evidence-based guidelines for surgical intervention are undefined.

The decision to treat CM-1 surgically may be based on a variety of considerations, such as anticipated syrinx improvement and stabilization of spinal cord damage, as well as the risk of postoperative complications. However, for many patients, disease-related symptoms are a primary motivation for surgery, making symptom-related quality of life (QOL) a key outcome metric for comparative effectiveness studies and one important factor in developing treatment guidelines. Therefore, standardized prognostic tools are needed to stratify preoperative disease severity based on the probability of postoperative improvement in QOL.

To date, multiple studies have attempted to identify predictors of long-term symptomatic outcome in CM-1 patients.^9–13 However, these investigations used heterogeneous outcome measures that were based on the research team’s impression of disease resolution. Moreover, these studies focused on the isolated significance of individual signs and symptoms, without providing an integrated system that can be applied uniformly to patients considering surgery. In order to improve patient counseling and to stratify patients for comparative studies, there is a pressing need for an integrated and predictive preoperative classification system. Thus, the objective of this study was to utilize clinical and neuroimaging features to predict long-term patient-reported QOL outcomes. The new Chiari Severity Index (CSI) is a preoperative grading system to predict long-term, patient-defined improvement from CM-1 surgery.

Methods

Patient Population

The Washington University in St. Louis Institutional Review Board approved all study procedures, and all patients and/or guardians provided informed consent. Patients included in this study received primary surgical treatment for CM-1 at St. Louis Children’s Hospital between January 2001 and July 2013, and had 5.5 months of follow-up or longer since surgery. We excluded patients that had undergone previous CM-1 surgery before 2001 or at an outside hospital, since we were unable to evaluate original presenting characteristics. We also excluded asymptomatic patients who underwent prophylactic treatment to prevent future neurological deficits but could not be expected to experience improvement in Chiari-related symptoms (e.g. asymptomatic syringomyelia).

We contacted eligible patients and asked them to complete a brief questionnaire via mail, using an online REDCap database,¹⁴ or in person during clinic visits. Follow-up phone calls and emails were used to increase response rates.

Data Collection

Preoperative signs and symptoms were recorded from retrospective medical record review. These findings were subsequently grouped into categories that were based on clinical relatedness and previously published classifications.^{9, 15} Thus, we collected information on the location and characteristics of headache symptoms, bulbar symptoms related to brainstem compression (central apnea, dysphagia, and nystagmus), as well as signs and symptoms related to spinal cord pathology. We grouped headaches into three categories: classic “Chiari headaches,” including occipital, valsalva-induced, post-tussive, and exertional headaches; frontotemporal headaches; and poorly-localized headaches (which included vertex, holocranial, and headaches not specified by specific location or characteristic). We defined myelopathic symptoms as: numbness; weakness; hyperreflexia; or unsteady gait. Paresthesias were not included in this group because such symptoms are frequently poorly-defined and often reported in the absence of definite spinal pathology. For instance, in our population, paresthesias were present in 28 patients, 17 of whom had no other myelopathic symptoms noted.

Beyond clinical characteristics, we recorded the following neuroimaging parameters potentially related to CM-1 pathology: 1) tonsillar ectopia, as the perpendicular distance from the tip of the cerebellar tonsil to McRae’s line;^{16, 17} 2) clivus-canal angle, as the angle between Wackenheim’s clivus baseline and the posterior C2 vertebral body line; ¹⁸ 3) dens angle, as the angle formed by a line through the C2 synchondrosis and a line through the odontoid tip measured from the posterior aspect of the C2 vertebral body;¹⁹ 4) pB-C2, as the perpendicular distance between the ventral dura and a line joining the basion to the infero-posterior C2 body;^{17, 19, 20} 5) obex position, as the distance between the obex and foramen magnum (basion-opisthion line);²¹ and 6) basilar invagination, as present when tip of the dens was greater than 5 mm above the Chamberlain’s line.¹⁸ We defined a syrinx as a contiguous spinal cord cavitation measuring at least 3 mm in maximum diameter on T2-weighted MRI. Patients with incomplete spine imaging but no syrinx noted on brain and/or cervical spine MRI (34/119) were designated as not having a syrinx.

Outcome Assessment

There is currently no validated, patient-based, disease-specific tool to evaluate CM-1 outcomes,²² and to assess the impact of an intervention on QOL requires the use of a static measure at pre- and post-intervention or the use of a transitional assessment after intervention.²³ Consequently, we evaluated patient-defined outcome by gauging responses to two brief questions: “How much of an improvement in your general health did you experience after your surgery?” and “How satisfied are you with your overall outcome from surgery?” To measure responses, we used a 5-point scale ranging from “no improvement” or “not at all satisfied” to “complete improvement” or “extremely satisfied.” In cases where patients were unable to answer independently, parents were instructed to answer with input from their child, as possible.

We defined patients as “improved” if they answered “a lot of improvement” or “complete improvement” to the first question and “very satisfied” or “extremely satisfied” to the second question. We chose this cutoff to define an “improved” response that seemed to unambiguously indicate a substantial benefit from surgery.

Statistical Analysis

Given the large number of imaging parameters assessed, we used bivariate analyses to screen for metrics potentially predictive of outcome. Chi-Square analysis or Fisher’s exact test was used to assess associations between categorical variables. Independent samples t-test or the Mann-Whitney U test was used to analyze continuous variables. We selected variables with p-values of 0.1 or less for multivariate analysis, with continuous variables dichotomized to maximize their prognostic value.

We analyzed the specified clinical variables and qualifying neuroimaging parameters using the multivariate technique of sequential sequestration.²⁴ Using this approach, the factor most strongly associated with outcome (i.e. predicting the worst prognosis) was identified first. Subsequently, the subgroup with this factor was removed from the study population, and the next most important characteristic associated with poor outcome in the remaining cohort was identified. The process was continued with the goal of achieving a set of ranked, ordinal categories that effectively predicted outcome. Compared to automated techniques, sequential sequestration involves judgment decisions when determining the ranked categories, helping ensure the final system is clinically reasonable.

To combine independent clinical and neuroimaging grading systems into a single index, we used the technique of conjunctive consolidation.²⁴ The goal of conjunctive consolidation is to “find and sequentially combine (or ‘consolidate’) the most important variables” impacting outcome.²⁴ Thus, cross-table analysis is used to assess the conjoined impact of two sets of variables (e.g. clinical and neuroimaging grades) to influence clinical outcome. Adjacent cells are then consolidated based on both statistical and clinical considerations (i.e. clinically reasonable groupings with similar outcome rates).²⁴ This process produces a final index that is mathematically powerful (i.e. separates groups with substantially different outcome rates) and also consistent with accepted clinical knowledge. Compared to multivariable logistic regression, conjunctive consolidation often performs as well in predictive modeling and is typically easier to apply into clinical settings.²⁵

The statistical significance of the clinical, neuroimaging, and integrated CSI were evaluated using the Cochran-Armitage test for linear trends. The discrimination of each system was evaluated using the c-statistic. Equivalent to the area under the receiver operating characteristic curve, the c-statistic ranges from 0.5 (no predictive ability) to 1.0 (perfect predictive ability).²⁶ All statistical analyses were done using SAS version 9.3 (SAS Institute, Cary, North Carolina). Two-tailed p<0.05 was considered statistically significant for all analyses.

Results

Of the 314 eligible patients contacted, 164 (52%) completed questionnaires. However, given that 7.6% of patients had mailings returned for wrong addresses and could not be reached otherwise, the true response rate among patients that received the mailing may have been higher. Six asymptomatic patients were excluded from the analysis, leaving a study cohort of 158 children, including 93 (59%) females. Most surgeries (74%) involved posterior fossa decompression with dural augmentation, with the remainder being extradural decompressions. One patient also underwent occipitocervical fusion at primary treatment. The median age at the time of surgery was 12.2 years (range 6 months to 18 years). The median follow-up period was 4.8 years (range 6 months to 12 years).

Overall, 111/158 (70%) of patients improved after surgery. Among patients with less than one-year of follow-up, 73% improved, compared to 79% of patients with 1–3 years follow-up, 67% with 4–7 years follow-up, and 61% with more than seven years of follow-up (p=0.06 for linear trend). Among the 68 patients 18 years or older at the time surveyed (assumed to have answered independently), 65% improved, compared with 74% of the patients younger than 18 years at the time surveyed (assumed to have answered with parental guidance) who improved (p=0.18; Chi-Square test).

Clinical Grading System

Thirty-nine of 158 patients with clinical data had no imaging available for review. In order to maximize the use of available information, we developed the clinical grading system using all patients (n=158) and developed the neuroimaging grading system and final CSI using the subset with neuroimaging data (n=119). For the clinical grading system, we focused on signs and symptoms commonly experienced by CM-1 patients—headaches and signs of brainstem or spinal cord pathology—with the goal of creating an index that was simple yet sufficiently comprehensive to be applicable to all CM-1 patients.

As shown in Figure 1, the presence of myelopathic symptoms predicted the worst outcome (58% improved); thus, these patients were isolated from the population first to form grade 3. Patients with frontotemporal headaches or no headaches had improvement rates slightly below baseline (64–67%) and thus were removed next (grade 2). After excluding patients with myelopathic symptoms, frontotemporal headaches, or no headaches, the remaining patients with classic Chiari headaches or poorly localized headaches had the best outcome (79–83% improved) and defined the grade 1 cohort. Only 21 patients experienced bulbar signs, and consideration of this category did not improve the model’s prognostic ability. Thus, the CSI clinical grading system had improvement rates ranging from 58% to 81% across categories and was statistically significant by a test for linear trend (z=2.57; p=0.01).

Figure 1.

A. The process of sequential sequestration used to develop the CSI clinical grading system. B. The final clinical grading system.

Neuroimaging Grading System

Given the large number of neuroimaging parameters examined, we performed bivariate analyses to screen for potentially influential predictors of outcome (Table 1). These analyses showed a significant negative association between the presence of a large syrinx ≥ 6 mm and postoperative improvement (p=0.049). In addition, these analyses suggested basilar invagination was associated with worse outcome (p=0.054). Other neuroimaging parameters, including tonsillar descent (p=0.23), clivus-canal angle (p=0.77), dens angle (p=0.13), obex position (p=0.95), and pB-C2 (p=0.87), were not associated with postoperative outcome.

Table 1.

The association between preoperative craniovertebral junction and syrinx measurements and postoperative improvement.

Measure	Improved (n=83)	Not Improved (n=36)	P-Value
Clival canal angle, median (IQR), deg	155 (16)	156.5 (17.5)	0.77
Dens angle, median (IQR), deg	75 (13)	78.5 (9)	0.13
Obex position, median (IQR), mm	3 (8)	4 (10)	0.95
pB-C2, median (IQR), mm	4 (3)	4 (2.5)	0.87
Tonsillar descent, median (IQR), mm	13 (9)	14.5 (7)	0.23
Syrinx present, n (%)	35 (69)	16 (31)	0.82
Syrinx ≥ 6 mm, n (%)	16 (55)	13 (45)	0.049
Basilar invagination, n (%)	3 (38)	5 (62)	0.054

The development of the neuroimaging grading system is shown in Figure 2. While basilar invagination was marginally associated with worse outcome (p=0.054), given the small number of patients affected (n=8), we lacked sufficient power to assess its independent prognostic importance. Consequently, presence of a large syrinx (≥ 6 mm) was the only predictor included in the neuroimaging grading system. Thus, patients with a large syrinx constituted grade B and had a 55% improvement rate, while those with a small syrinx or no syrinx were included in grade A and had a 74% improvement rate. This difference in improvement was statistically significant (z=1.965; p=0.049).

Figure 2.

A. The process of sequential sequestration used to develop the CSI neuroimaging grading system. B. The final neuroimaging grading system.

Consolidation of Clinical and Neuroimaging Grading Systems

While the clinical and neuroimaging grading systems independently predicted treatment outcome, the goal of this study was to produce a system that integrated both clinical and neuroimaging parameters. Therefore, we used conjunctive consolidation to develop the Chiari Severity Index. The process of conjunctive consolidation used to develop the CSI is shown in Figure 3. CSI grade 1 consisted of patients in clinical grade 1, regardless of neuroimaging findings. CSI grade 2 included patients in groups 2A (clinical grade 2; imaging grade A) and 3A, while CSI grade 3 consisted of the remaining patients in 2B and 3B.

Figure 3.

A. The process of conjunctive consolidation used to integrate the CSI clinical and neuroimaging grading systems. B. The final Chiari Severity Index.

The performance of the CSI was compared with the independent clinical and neuroimaging grading systems in Table 2. All models were statistically significant by a test for linear trend (p<0.05), and all three systems had a monotonic outcome gradient (i.e. the difference in improvement between grades 1 and 3) across grades. However, the outcome gradient was notably higher for the integrated CSI (38%) than for the clinical (23%) or neuroimaging (19%) systems. The c-statistic for the CSI (0.66) also was moderately higher than that for either the clinical (0.62) or neuroimaging (0.58) system, indicating greater discrimination using the integrated CSI.²⁶

Table 2.

Comparison of the performance of the clinical and neuroimaging grading systems with the integrated Chiari Severity Index.

Model	Proportion Improved	Outcome Gradient	P-value for Trend	Discrimination, C-statistic
Clinical Grading		23	0.01	0.62
1	52/64 (81%)
2	40/61 (66%)
3	19/33 (58%)
Neuroimaging Grading		19	0.049	0.58
A	67/90 (74%)
B	16/29 (55%)
Chiari Severity Index		38	0.002	0.66
1	38/46 (83%)
2	35/51 (69%)
3	10/22 (45%)

Discussion

In this study, we used the process of sequential sequestration to identify key clinical and neuroimaging features to develop novel clinical and neuroimaging grading systems for CM-1 patients. We then used conjunctive consolidation to integrate these independent systems to form the Chiari Severity Index. The rates of improvement, according to different CSI grades, ranged from 83% improvement in grade 1 to 45% in grade 3, and these differences were statistically significant predictors of patient-reported clinical outcome.

Since at least 1976, multiple groups have tried to predict which CM-1 patients are most likely to benefit from surgery, ^{9–13, 27–29} and in 1995, Bindal and colleagues proposed a classification system that categorized patients based on the presence of syringomyelia and signs of brainstem compression.¹¹ However, the proposed categories provided limited detail, and with only 27 patients, this study had little power to develop or evaluate a grading system.

Beyond small sample size, previous attempts to identify prognostic factors focused on the isolated importance of various clinical or imaging findings. Indeed, other than Bindal and colleagues study based on a small number of patients, we are not aware of any integrated clinical and neuroimaging systems to predict CM-1 treatment outcomes. In addition, previous investigations evaluated outcome based on medical record review, which may not adequately reflect patient perspectives of their symptomatic improvement.^{9–12, 27–29}

Recognizing these shortcomings, we developed clinical and neuroimaging grading systems and an integrated CSI. Worrisome clinical findings (i.e. myelopathic symptoms, frontotemporal headache, or no headache) were usually present in patients that did not improve; conversely, patients with isolated classic Chiari or poorly localized headaches typically responded well to surgery. Myelopathic symptoms, in particular, likely indicate neurological damage not always responsive to surgery, while frontotemporal headaches or the absence of headache likely reflect the presence of bothersome symptoms not necessarily related to CM-1.

Using conjunctive consolidation, we integrated these clinical findings with the prognostic information provided by neuroimaging. Conjunctive consolidation has been used to develop grading systems for lung cancer,³⁰ head and neck cancer,^{31, 32} obstructive sleep apnea,²⁵ and rhinosinusitis.³³, and is particularly useful for combining independent indices into a composite measure. Based on the conjunctive consolidation approach, neuroimaging information did not alter outcome predictions for patients with favorable clinical presentations. Thus, patients with isolated Chiari or poorly localized headaches who are treated before clinical manifestations of spinal cord pathology develop typically experience meaningful symptomatic improvement, even in the presence of a large (≥ 6 mm) syrinx. However, for patients with more worrisome clinical presentations, presence of a large syrinx predicted substantially lower improvement rates. Thus, a large syrinx—compared to any syrinx, which has been the focus of most previous reports—may indicate severe symptomatic spinal cord damage unlikely to improve.^{9–11, 34, 35} In addition, the presence of a large syrinx may lead some patients with frontotemporal headache or no headache to receive surgery, despite the absence of clinical symptoms directly attributable to CM-1, potentially explaining their lower improvement rates.

With the exception of basilar invagination, craniovertebral junction parameters, including tonsillar ectopia, were not associated with long-term outcome. This finding is consistent with results from other groups and suggests that such metrics likely should not be used to predict symptomatic treatment outcome.^{10, 19, 29} In addition, since only one patient underwent occipitocervical fusion with primary treatment, it seems unlikely that this additional intervention masked a potential relationship between craniovertebral junction metrics and improved QOL. However, given this low rate of fusion, our ability to comment on the relationship between frank occipitocervical instability and improvement in QOL after treatment is limited and should be explored in future work. Similarly, while basilar invagination was marginally associated with lower improvement rates, we were unable to include this metric in the CSI due to the small number of patients affected. Therefore, future studies with larger sample sizes should investigate whether this parameter improves the prognostic ability of the CSI.

While the CSI should aid physician-patient discussions and is an important step in developing evidence-based treatment guidelines, it is not itself a treatment protocol, and it should be applied with consideration of other important outcomes in CM-1 research. For instance, while patient-defined QOL is likely to be a primary endpoint for most comparative effectiveness studies, radiologic syrinx improvement and stability of neurological deficits (i.e. myelopathic symptoms) are also key considerations. Thus, the CSI should not replace traditional indications for surgery, such as preventing progressive neurological damage in patients with symptomatic syringomyelia that may not experience substantial symptomatic improvement. Finally, CM-1 treatment guidelines will likely consider postoperative complications, which are known to vary widely based on surgical technique,³ but are not addressed by the CSI.

Limitations

Beyond these considerations, this study also has limitations. First, the CSI was developed using pediatric patients from a single institution. Therefore, external validation in other centers and in adult populations is necessary. Second, while we did not find a significant trend in improvement rates based on duration of follow-up, our results suggested improvement rates may be lower when assessed farther from surgery. However, this may be due to recall bias, as we were unable to assess changes in health perceptions (i.e. QOL) over time. Therefore, future work should prospectively survey patients before and after CM-1 surgery to explore the relationship between preoperative CSI and QOL improvement at varying durations of follow-up. Third, although our sample size was large, our ability to assess rarer findings, such as basilar invagination and specific bulbar symptoms, was limited. Thus, future work in larger populations may explore whether consideration of such characteristics improves the prognostic ability of the CSI. In addition, assessment of preoperative signs and symptoms was based on retrospective chart review, which may have been influenced by incomplete or inconsistent recording of information in the medical record. Furthermore, while the response rate in this study was good, there is also the potential for non-response bias, which is a common concern in survey research. Finally, as more rigorous QOL outcome measures for CM-1 are developed, the performance of the CSI should be tested against these novel metrics.

Conclusion

In this study we developed the CSI, a novel severity index to evaluate CM-1 patients considering surgery. Integrating both clinical and neuroimaging features, the CSI identifies which patients are most likely to improve following surgical treatment. Implementing the CSI into clinical practice will enable more evidence-based preoperative counseling, and applying the CSI in comparative studies will improve the quality of Chiari outcomes research.