Case Misclassification in Studies of Spinal Manipulation and Arterial Dissection

Abstract

Goals

Spinal manipulation has been associated with cervical arterial dissection and stroke but a causal relationship has been questioned by population-based studies. Earlier studies identified cases using International Classification of Diseases-9 codes specific to anatomical stroke location rather than stroke etiology. We hypothesize that case misclassification occurred in these previous studies and an underestimation of the strength of the association. We also predicted that case misclassification would differ by patient age.

Materials and methods

We identified cases in the Veterans Health Administration database using the same strategy as the prior studies. The electronic medical record was then screened for the word “dissection.” The presence of atraumatic dissection was determined by medical record review by a neurologist.

Findings

Of 3690 patients found by International Classification of Diseases-9 codes over a 30-month period, 414 (11.2%) had confirmed cervical artery dissection with a positive predictive value of 10.5% (95% CI 9.6–11.5%). The positive predictive value was higher in patients <45 versus ≥45 years (41% vs. 9%, p<0.001). We reanalyzed a previous study which reported no association between spinal manipulation and cervical artery dissection (OR=1.12, 95% CI 0.77–1.63), and re-calculated an odds ratio of 2.15 (95% CI 0.98–4.69). For patients under age 45, the OR was 6.91 (95% CI 2.59–13.74).

Conclusions

Prior studies grossly misclassified cases of cervical dissection and mistakenly dismissed a causal association with manipulation. Our study indicates that the odds ratio for spinal manipulation exposure in cervical artery dissection is higher than previously reported.

INTRODUCTION

Spinal manipulation therapy (SMT) is administered to 8% of American adults annually¹. It is associated with adverse neurological outcomes including cervical artery dissection(CAD) and stroke^2–4. The magnitude of risk has been estimated at a high of 1 in 958 manipulations⁵ to as low as 1 in 5.85 million manipulations⁶. The causal link between SMT and CAD has been questioned⁷. In 2001, a large case-control study by Rothwell et. al. demonstrated an association between posterior circulation stroke and chiropractic visits in patients <45 years of age but found no relationship in those ≥45 years⁸. In 2008, Cassidy et. al. replicated Rothwell’s results and also demonstrated an association between case status and visits to primary care physicians (PCP). Cassidy suggested that the observed associations are due to reverse causation bias whereby patients with dissections seek treatment from chiropractors or PCPs for dissection-related symptoms like neck pain.

In the earlier studies, cases were identified by using International Classification of Diseases Ninth Revision (ICD-9)codes that were specific for a neurovascular location (posterior circulation) rather than codes for a vascular diagnosis (dissection). As a result, they likely classified patients with stroke due to conventional mechanisms described in posterior circulation registries as their cases. Assuming that this case misclassification was random with respect to SMT exposure, it is likely that both the Rothwell and Cassidy studies underestimated the association between dissection risk and SMT⁹. Furthermore, it is known that patients with vascular risk factors will have more frequent contact with their PCPs¹⁰. If the cases in the Cassidy study were mostly patients with atherosclerosis then an association with PCP visits is expected. Finally, given the higher prevalence of dissection as a stroke mechanism in younger patients (those under the age of 45) and the increased prevalence of atherosclerosis with age over 40^11–13, we hypothesized that the extent of case misclassification would differ by patient age, with older patients more likely to be misclassified than younger ones. We sought to evaluate the magnitude of case misclassification in the Rothwell/Cassidy studies by employing their ICD-9 based case identification strategy followed by refined case assessment with detailed medical record review to identify those with true CAD overall and within age strata(<45 years and ≥45years).

MATERIALS AND METHODS

By accessing the encounter diagnosis table in the clinical data warehouse we identified all patients in the Veterans Health Administration (VA) electronic medical record(EMR), a population of 15,779,020 veterans, with ICD-9 codes used by Rothwell/Cassidy studies for the period January 2009 to August 2011^{8, 14}(Table 1). The earlier studies omitted the dissection specific codes (443.xx) in their case definition because they were not in use in Ontario at the time (personal communication, Navin Goocool, April 30, 2013). The population in our study did have these codes available and therefore in order to avoid an overestimation of case misclassification, we included the 3 additional dissection codes in our initial EMR query (“modified Rothwell/Cassidy strategy”). The entire record of each patient associated with one of those eight ICD-9 codes was then searched for the presence of the word “dissection” in the EMR using Medical Domain Web Services. Available sources included discharge summaries, radiology reports, consultation notes, out patient records, and any other record containing text. A study physician then reviewed the extracts from the EMR that included the word “dissection” to determine whether a vertebral or carotid dissection had been diagnosed. The adjudication was supplemented by reviewing neuroimaging studies and other EMR records as needed. Data collected included patient age at the time of the index event and the location of a dissection if present (vertebral, carotid, or both). The definition of atraumatic CAD used during the record review was: a clinical presentation consistent with dissection, no competing stroke diagnoses, and confirmation of dissection following appropriate confirmatory investigations. Atraumatic was defined as: not associated with vertebral fracture in the cervical spine. Clinical presentation consistent with dissection includes any of the following: asymptomatic, sudden onset meningismus secondary to subarachnoid hemorrhage, new onset headache or asymmetric neck pain, lower cranial neuropathy, Horner syndrome, cerebral or retinal ischemia (TIA or stroke). We estimated the proportion of ICD-9-detected cases likely to be true CAD events (positive predictive value) with corresponding 95% confidence intervals in the entire population and within strataby age (<45 and ≥45 years). Information about exposure to SMT for individual patients was not available in the VA database and was not collected.

Table 1.

ICD-9 Codes and Definitions

A. Codes used in Rothwell/Cassidy studies	Definition
433.00	Occlusion and stenosis of basilar artery without cerebral infarction
433.01	Occlusion and stenosis of basilar artery with cerebral infarction
433.20	Occlusion and stenosis of vertebral artery without cerebral infarction
433.21	Occlusion and stenosis of vertebral artery with cerebral infarction
900.9	Injury to unspecified blood vessel of head and neck
B. Additional dissection-specific codes
443.21	Dissection of carotid artery
443.24	Dissection of vertebral artery
443.29	Dissection of other artery

In order to anticipate the impact of misclassification on prior epidemiologic studies of SMT and CAD, we conducted a sensitivity analysis by applying the positive predictive value measured in the VA data to aggregated data reported in the Cassidy study¹⁴ both 1) across the entire cohort and 2) within strata defined by age (<45 years and age ≥45 years). We did not measure the negative predictive value and assumed it to be 100%, i.e. it is extremely unlikely that the word “dissection” would fail to appear anywhere in the EMR text of an ICD-9-identified stroke patient who had been diagnosed with CAD by VA physicians. Since we did not have access to individual data on the SMT exposure of each case and control, we assumed the misclassified cases (ICD-9 code positive but CAD negative) had the same exposure rate as the control population (3.95%). The SMT exposure rate in the true CAD cases was calculated assuming that the case exposure rate in the original report represents a weighted average of SMT exposure rates in true and misclassified cases. Odds ratios and corresponding 95% confidence intervals for the association between SMT and CAD were calculated based on these assumptions.

Statistical analyses used the SAS statistical package, version 9.2 (SAS Institute, Cary, NC) and R (version 2.15.1). All p-values are 2-sided. The Institutional Review Boards of the VA Boston Healthcare System and Tufts Medical Center approved the study.

RESULTS

We identified 3690 unique apparent cases in the VA population using the modified Rothwell/Cassidy strategy(Table 2) within the study period. Of those, 1066 (28.9%) patients had the word “dissection” somewhere in their EMR. Of these patients, 414 (11.2%) had CAD confirmed by subsequent review. Twenty-six patients were excluded due to a concomitant vertebral fracture leaving 388 patients with confirmed atraumatic CAD, corresponding to a positive predictive value for the modified Rothwell/Cassidy strategy of 10.5% (95% CI: 9.6– 11.5%). The majority (96.0%, n=3544) of patients identified in this study were ≥45 years of age, with only 146(3.9%) patients under 45 years. There was a statistically significant difference in the positive predictive value in patients <45 years compared to those ≥45 years (41.1% vs.8.8% respectively, p<0.001).

Table 2.

Clinical Characteristics of Patients Identified Using ICD-9 Codes

		Cases (n = 3690)
Patients with “Dissection” in EMR, n (%)		1066 (28.9)
	433.00	155 (4.2)
	433.01	53 (1.4)
	433.20	321 (8.7)
	433.21	344 (9.3)
	443.24	107 (2.9)
	443.29	74 (2.0)
	900.9	12 (0.4)
Patients with Confirmed CAD, n (%)		414 (11.2)
Patients with Confirmed Atraumatic CAD (Positive Predictive Value (%), 95% CI)		388 (10.5, 9.6 – 11.5)
CAD by Type, n (%)
	Vertebral	141 (36)
	Carotid	240 (62)
	Both	7 (2)
ICD-9 code identified cases by Age Group, n (%)
	<45	146 (3.9)
	≥45	3544 (96.0)*
Patients with Confirmed Atraumatic CAD by Age Group (Positive Predictive Value (%), 95% CI)
	<45	60 (41.1, 33.5–49.2)
	≥45	313 (8.8., 7.9–9.8)

^*These results assume that the age distribution of patients without the word dissection (n=2624) is the same as in the 1066 charts that were reviewed with known age information.

As a sensitivity analysis, were analyzed Cassidy’s data across the entire cohort and within age strata (<45 and ≥45 years)by applying the positive predictive values as measured in the VA population. Assuming the SMT exposure rate in misclassified cases to be the same as population controls (3.95%)then the corresponding odds ratio between SMT and CAD is 2.15 (95% CI 0.98–4.69), as compared to the null association (1.12; 95% CI 0.77–1.63), which was reported (Table 3B). Among the subgroup of the population under 45 years of age and applying the above assumptions, those with a chiropractor visit within 30 days of their stroke would have nearly 7 times the odds of CAD (OR = 6.91, 95% CI 2.59–13.74).

Table 3.

Re-analysis of association between chiropractic exposure and case status from Cassidy et al after exclusion of proportion of cases unlikely to be CAD

Table 3A: Association between chiropractic (DC) and case status (data from Ref 4)
	Entire Cohort		Age < 45		Age ≥ 45
	Cases	Controls	Cases	Controls	Cases	Controls
DC visit in 30 days	36	125	13	18	23	107
No DC visit in 30 days	782	3039	89	390	693	2649
Total	818	3164	102	408	716	2756
Odds Ratio (95% CI)	1.12 (0.77, 1.63)		3.16 (1.50, 6.70)		0.82 (0.52, 1.30)

Table 3B: Re-analysis of Table 3a after exclusion of proportion of cases unlikely to be CAD*
	Entire Cohort		Age < 45		Age ≥ 45
	Cases	Controls	Cases	Controls	Cases	Controls
DC visit in 30 days	7	125	10	18	0	107
No DC visit in 30 days	82	3039	32	390	66	2649
Total	89	3164	42	408	66	2756
Odds Ratio (95% CI)	2.15 (95% CI 0.98–4.69)		6.91 (95% CI 2.59–13.74)		Cannot calculate

^*ICD-9 code positive predictive value measured in the VA data applied across the entire cohort and within strata defined by age (<45 years and age ≥45 years). The misclassified cases (ICD-9 code positive but CAD negative) assumed to have the same SMT exposure rate as the control population (3.95%). The SMT exposure rate in the true CAD cases was calculated assuming that the case exposure rate in the original report represents a weighted average of SMT exposure rates in true and misclassified cases. Counts in the table were rounded to the nearest whole integer.

DISCUSSION

In a large population, using the same case identification strategy as reported in previous population-based studies, only 10.5% of cases identified as cases by ICD-9 codes were found to have an atraumatic cervical artery dissection. As predicted, this case misclassification was greater in patients over 45 years of age.

Our findings suggest that cases identified by posterior circulation anatomical ICD-9 codes are mostly strokes caused by non-dissection etiologies, which would attenuate the association with SMT. These misclassified cases are likely to be caused by conventional stroke mechanisms such as atherosclerosis and cardiac disease, as described in large registries of posterior circulation strokes¹⁵. Cassidy suggested that the association between cases and PCP/SMT exposure was due to patients with preexisting dissections seeking care for neck pain (reverse causation). However, if the ICD-9 code positive predictive value measured in the VA database is generalizable to the Ontario health system data, then the Cassidy study actually found an association between PCP visits and conventional stroke patients with atherosclerotic and cardioembolic strokes. This association is well known and has been described before. It is due to the frequent clinical visits needed to manage established vascular risk factors¹⁰. Our sensitivity analysis suggests that the odds ratios for the association between SMT and CAD would be very large with accurately identified cases. Lastly, the misclassification may disproportionately affect odds ratios for those under 45 years old – a group of patients with a lower prevalence of atherosclerosis-related infarcts and a higher prevalence of strokes due to dissections¹⁶ Given the small numbers of true cases, odds ratios within age strata could not be calculated, but our sensitivity analysis suggests the association between SMT and CAD in younger patients is markedly stronger after adjusting for case misclassification.

A strength of this study is our access to a large population (>15 million people), which permitted study of a relatively rare event. Our study has several limitations. First, previous Canadian studies with different demographics, administrative and coding practices may limit the generalizability of results from the VA. There are more males in the VA population (~90%¹⁷) than in the general population of Ontario (51% male¹⁸). This may limit the direct comparison of incidence rates because males are diagnosed with CAD more often than females by a ratio of roughly 2:1¹⁹. This was also seen in the Ontario studies. The higher prevalence of males in the VA population may be expected to lower the calculated OR in Table 3B because of a 50% higher exposure by women to chiropractic²⁰. However, CAD incidence is not the focus of this analysis and we are not aware that coding of CAD differs by gender. Therefore, the very low positive predictive value of the Rothwell/Cassidy strategy of case selection, the main finding of this study, is not likely to be affected by this difference in male/female distribution. Second, only one neurologist was responsible for the medical record review and we have no data on inter- or intra-rater reliability. However we point out that most of the paring down of “cases” was done by searching the EMR for the single word “dissection” which excluded 71% of ICD 9-identified cases without requiring any individual’s judgment. So even with disagreement between observers, the overall result would only be marginally affected. Third, we could not directly calculate the negative predictive value of ICD-9 code identification of CAD in the VA. We assume that it was 100% noting that it is highly unlikely for a patient to have an identified dissection with no use of the word in any text field. Finally, our sensitivity analysis was conducted without access to individual patient data so assumptions about SMT exposure rates in true and misclassified cases are unverifiable.

If our estimates of case misclassification are applicable outside the VA population, odds ratios for the association between SMT exposure and CAD are likely to be higher than those reported using the Rothwell/Cassidy strategy, particularly among younger populations. Future epidemiologic studies of this association should prioritize the accurate classification of cases and SMT exposure.