Predictive Value of International Classification of Disease Codes for Idiopathic Intracranial Hypertension (IIH) in a University Health System

Abstract

Introduction:

Misclassification bias is introduced into medical claims-based research by reliance on diagnostic coding rather than full medical record review. We sought to characterize this bias for idiopathic intracranial hypertension (IIH) and evaluate strategies to reduce it.

Methods:

Retrospective review of medical records was conducted using a clinical data warehouse containing medical records and administrative data from an academic medical center. Patients with one or more instances of international classification of disease (ICD) 9 or 10 codes for IIH (348.2 or G93.2) between 1989 and 2017 and original results of neuro-imaging (head CT or MRI), lumbar puncture and optic nerve examination were included in the study. Diagnosis of IIH was classified as definite, probable, possible or inaccurate based on review of medical records. Positive predictive value (PPV) for IIH ICD codes was calculated for all subjects, subjects with an IIH code after all testing was completed, subjects with high numbers of IIH ICD codes and codes spanning longer periods of time, subjects with IIH ICD codes associated with expert encounters (ophthalmology, neurology or neurosurgery), and subjects with acetazolamide treatment.

Results:

Of 1005 patients with ICD codes for IIH, 103 had complete testing results and were included in the study. PPV of ICD-9/10 codes for IIH was 0.63, PPV in restricted samples were 0.82 (code by ophthalmologist n=57), 0.70 (acetazolamide treatment n=87), 0.72 (code after all testing, n=78). High numbers of code instances and longer duration between first and last code instance also increased PPV.

Conclusion:

An ICD 9 or 10 code for IIH had a positive predictive value of 63% for probable or definite IIH in patients with necessary diagnostic testing performed at a single institution. Coding accuracy was improved in patients with an IIH ICD code assigned by an ophthalmologist. Use of coding algorithms considering treating providers, number of codes and treatment are a potential strategy to reduce misclassification bias in medical claims-based research on IIH. However, these are associated with reduced sample size.

Introduction

Idiopathic intracranial hypertension (IIH) causes disability and reduced quality of life due to headaches and vision loss, which is permanent and at the level of blindness in a small, but significant number of affected individuals.(1) The prevalence is increasing with associated increased burden on health care systems around the world.(2) Proposed risk factors identified based on associations with IIH in case reports, case series and case-control studies, include female sex, obesity and certain medications.(3) However, many of these have not been confirmed in a population sample. A big data approach using medical claims data has potential applications to supporting these as causal associations because they capture real world experience and have relatively large sample sizes compared to traditional epidemiologic study approaches.(4)

An important part of any medical study is identification of cases. Medical claims data contain information about medical encounters including type, diagnostic codes (e.g. International Classification of Disease (ICD) versions 9 and 10) and procedure codes (e.g. Current Procedural Terminology (CPT)) that can be leveraged to identify cases. For example, IIH cases might be identified as those with an encounter including the ICD-9 diagnostic code 348.2 or ICD-10 diagnostic code G93.2. However, claims data lacks the detailed medical records necessary to do this based on diagnostic test and examination results and therefore may lead to misclassification bias. This was demonstrated in a study of IIH patients receiving care in the emergency room at an academic medical center where only 55% of charts with the ICD-9 code for IIH met diagnostic criteria on full chart review.(5) More detailed algorithms have applied additional inclusion criteria such as requiring a diagnostic code for IIH being filed after completion of necessary testing.(6) However, the accuracy of such algorithms has not been evaluated.

The objective of this study is to compare the accuracy (positive predictive value) of IIH case identification algorithms for patients receiving care in a university health system using medical claims data.

Methods

This is a retrospective study performed using the Stanford Research Repository (STARR), which is a clinical data warehouse containing medical records and administrative data for patients receiving care at Stanford Health Care from 1995 to present. Approval for this study was granted by the Stanford Office for Research with a waiver of informed consent.

Subjects for this study were those who had an ICD code for IIH and records of completing necessary testing to confirm the diagnosis within the health system. Potential subjects were those with one or more encounters associated with ICD codes for IIH (ICD-9 348.2 or ICD-10 G93.2) prior to July, 2017, which was the date that we commenced this retrospective study. Inclusion criteria were completion of lumbar puncture (LP) with available report (CPT codes 62270, 62272, ICD-9 03.31, ICD-10 009U3*), completion of neuro-imaging (CT head (CPT 70450, 70460, 70470) or MRI brain (70551, 70553) based on manual review of radiology records) with available report occurring within 12 months of lumbar puncture and documentation of optic documented nerve exam (search for “fundus exam, eye exam or optic nerve”) within 12 months of lumbar puncture.

Medical records of included subjects were used as the basis for diagnosis classification. We used the modified Dandy criteria as used in the Idiopathic Intracranial Hypertension Treatment Trial (IIHTT), stratified based on revised pseudotumor cerebri diagnostic criteria into definite, probable and possible.(7, 8) All IIH diagnoses required lack of secondary causes of high ICP on neuroimaging, CSF analysis and medication history. Definite IIH was diagnosed for LP opening pressure > 25 cm H₂O and papilledema or optic atrophy on fundus exam. Probable IIH was diagnosed for LP opening pressure 20–25 cm H₂O with optic nerve edema on exam or findings of high ICP noted in neuroimaging reports (e.g. empty sella, globe flattening, increased CSF in optic nerve sheath) or if opening pressure was not recorded, optic nerve edema on exam responsive to therapy. Possible IIH was diagnosed in subjects with LP opening pressure > 25 cm H₂O without papilledema, optic atrophy, or findings of high ICP noted in neuroimaging reports. Subjects on treatment for IIH at time of normal eye exam were classified as possible (on treatment). Subjects not meeting criteria for IIH were classified as not having IIH and alternative diagnoses for their symptoms was recorded. Accuracy of case identification for the entire sample was calculated as the positive predictive value (# IIH/total # subjects).

Accuracy of case identification for patients seen by expert specialists was calculated using the sub-groups of subjects with an IIH code associated with an ophthalmology, neurology or neurosurgery encounter. Accuracy of case identification for patients with the appropriate diagnostic sequence was calculated using the sub-groups of subjects with at least one diagnostic code for IIH either on the same day as completion of necessary testing (i.e neuro-imaging and lumbar puncture) or on a subsequent day. Accuracy of case identification for patients treated with acetazolamide was calculated using the sub-group of subjects with an acetazolamide prescription in the medical record.

The impact on continuity of care for IIH on accuracy of case identification was studied using duration (in years) between initial and most recent IIH associated encounters, the total number of IIH code instances and the number of unique days with IIH codes. Case identification accuracy was calculated within subject groups defined by quartiles of these continuous variables.

Results

1005 potential subjects with one or more instances of ICD-9/10 codes for IIH (348.2 or G93.2) between 1995 and 2017 were identified. 300 of these had lumbar punctures with reports. Of these, 55 lacked neuro-imaging within the necessary time frame, 105 lacked optic nerve exams and 37 lacked both. 103 subjects were included in the study (Table 1). Of 65 subjects with IIH, all definite cases (n=39) and 12/15 probable cases had papilledema. 3 probable cases had unknown papilledema status because eye exam was documented after treatment, but were included as probable due to neuro-imaging findings. 3/12 possible cases lacked papilledema at time of diagnosis and 9/12 did not have an eye exam documented until after treatment. There were no patients with papilledema with ICP < 20 cm H₂O. Subjects without IIH had primary headache syndrome (12), metastatic or primary brain tumors (9), meningitis (5), venous sinus thrombosis (3), intracerebral hemorrhage (3), inflammation (2) and 1 each of hydrocephalus, medication induced IH, CSF leak and encephalocele.

Table 1:

Subjects with at ICD9 or 10 coding for IIH, lumbar puncture, neuro-imaging and optic nerve exam at an academic medical center (n=103)

Variable	Distribution
Age	36.6 +/− 13.9 years
Sex	76 (74%) female
LP opening pressure
Elevated (> 25 cm H2O)	63 (61%)
Borderline (20–25 cm H2O)	14 (14%)
Normal (< 20 cm H2O)	13 (13%)
missing	13 (13%)
CSF analysis
normal	54 (52%)
likely normal*	18 (17%)
abnormal	22 (21%)
missing	9 (9%)
neuro-imaging
normal	49 (48%)
ICP associated findings	19 (18%)
Abnormal – unrelated findings	17 (16%)
Abnormal – secondary cause	18 (17%)
Optic disk examination
Edema	54 (52%)
Atrophy	5 (5%)
Other optic disc findings	1 (1%)
Normal	43 (42%)
Diagnosis
IIH	65 (63%)
Definite	39 (38%)
Probable	15 (15%)
Possible (no papilledema on treatment)	8 (8%)
Possible (no papilledema prior to treatment)	3 (3%)
Not IIH	38 (37%)

^*isolated elevation in RBC or protein

Overall case identification accuracy for subjects with ICD-9 or 10 codes for IIH with necessary testing was 63.1%. This was increased to 71–82% when only subjects with codes associated with expert specialists (neurosurgery, neurology, ophthalmology) were considered (Table 2). Consideration of medical treatment with acetazolamide or appropriate diagnostic sequence with an IIH code given simultaneous with or following necessary diagnostic testing slightly improved case identification accuracy (69–72%, table 3). Amount of IIH codes either by time or number improved case identification accuracy to 83–92% for the upper quartiles of these variables (table 4).

Table 2:

Coding accuracy for IIH codes associated with encounters with relevant medical subspecialties

Specialist	n	Accuracy (PPV)
Neurology	52	0.75
Ophthalmology	57	0.82
Neurosurgery	17	0.71

Table 3:

Coding accuracy for IIH codes associated with appropriate medical management

Patient management	n	Accuracy (PPV)
Diagnostic sequence
Same day	87	0.69
Separate day	78	0.72
Acetazolamide treatment	87	0.70

Table 4:

Association between duration and volume of ICD9/10 codes for IIH and coding accuracy

	n	PPV
Instances of code
1–2 codes	28	0.36
3–8	24	0.63
8–21	26	0.65
> 21	25	0.92
Unique days with code
1 days	23	0.30
2–5	28	0.61
6–15	26	0.73
> 15	22	0.85
Duration with diagnosis code
1 year	48	0.43
2	21	0.76
3	10	0.80
> 3	24	0.83

Discussion

Medical coding using ICD and CPT codes, along with pharmacy records, constitute a map of health care delivery. The entries associated with individuals track their route through this map. In addition to applications for medical billing, analysis of this data can provide important insights into disease risk factors and delivery of care. Understanding the accuracy of coding is critical to development and interpretation of such studies. In the current study we assessed the accuracy of ICD 9 & 10 coding for IIH at an academic medical center and evaluated strategies to improve accuracy through sample selection criteria. We find that overall accuracy is 63% for one or more instances of ICD code for IIH in patients with neuro-imaging, lumbar puncture and fundus examination. Tightening criteria to include coding by a specialist, diagnostic sequence, medical treatment, number of codes or duration of codes all increased the accuracy of ICD codes for prediction of IIH. These also all reduced the sample size of collected cases. These results have relevance for design and interpretation of claims based medical research on IIH.

Diagnostic accuracy of IIH codes in our study is similar to that reported in two prior studies. A study of ER and inpatient utilization by IIH patients visits at a single institution reported accuracy of IIH ICD-9 code of 55%.(5) Similarities between this study and ours include location (single US academic medical centers). Differences include inclusion of patients without ER or inpatient care in the current study and consideration of external health records in diagnostic categorization of the prior study. A study using the national patient register in a Swedish county reported accuracy of IIH ICD-10 code as 65%.(9) In contrast to our study, this was a non-American population based study not limited to a single institution. Multiple factors likely contribute to this low accuracy including inter-provider variability in assigning codes, assignment of codes by administrative staff and transcription errors.(10) The errors may be administrative(11) or may reflect true diagnostic errors on behalf of the treating providers.(12)

A strategy to improve case identification from medical claims data is to modify inclusion and exclusion criteria for case selection beyond a single ICD code. We found that tightening sample selection based ICD code associated with specialist encounter (neurology, ophthalmology or neurosurgery), ICD code on same or later day as diagnostic test completion, acetazolamide treatment, more instances of ICD code (exclude < 3, include > 21), more unique days with ICD code (>5 days) and longer duration between initial and most recent code (>1 year) to each improve classification accuracy. However, each of these also reduced the sample size of included cases and the tradeoff between accuracy and sample size needs to be considered in applications of these findings.

Duration of code, coding by an ophthalmologist and diagnostic sequence are factors that improved IIH coding accuracy in our study that have not previously been considered. Sundholm et al considered age, sex, number of times IIH code was recorded, visit with a neurologist and acetazolamide prescription to develop a coding algorithm.(13) An algorithm considering age and 3 or more instances of IIH ICD-10 code increased accuracy to greater than 80% from 65% in a development sample and was confirmed in a validation sample.(13) The other variables were found not to be helpful in case identification. Sohdi et al. used an inclusion criteria of CPT codes for neuro-imaging and lumbar puncture within 15 days of the IIH code to identify IIH cases.(6) Mollan et al used an exclusion criteria ICD-10 codes for secondary causes of high ICP (e.g. hydrocephalus, cerebral venous sinus thrombosis, brain cancer and hypertensive encephalopathy),(2) a strategy in line with validated case identification strategies for other rare diseases.(14)

Beyond true diagnostic error, a challenge in both clinical care for and research about IIH is debate regarding the spectrum of this entity. Per its title IIH requires intracranial hypertension that is idiopathic. However, diagnostic criteria move beyond these two basic criteria to address the challenges in accurately measuring intracranial pressure and to select for people with pathophysiological effects from high ICP since 2.5% of normal adults have ICP > 25 cm H₂O(15) while 10% of those with clear pathological effects of elevated ICP (i.e. papilledema) have ICP ≤25 cm H₂O.(16) Diagnostic criteria considering papilledema, sixth nerve palsy and radiologic features narrow the diagnosis to those with measurable effects of high ICP. Classification schema that allow for definite, probable and possible cases, such as the one applied in this paper, allow for consideration of the broader spectrum of disease.(8) A limitation is likely false positive diagnoses, particularly in the possible category.

The narrow inclusion criteria, requiring primary documentation of necessary testing available in the medical record of the study center, limited the sample size, but accurately reflect the approach taken in claims based IIH analysis, which is case selection based on diagnosis (ICD) and testing (CPT) codes.(6) Though our sample size is on the same order of magnitude as prior studies,(5, 13) it is discouraging that only 103 subjects were identified from over 1000 patients with an IIH code in the medical record of the study center. The approximately 900 excluded patients likely fall into 3 categories: inaccurate diagnosis, incomplete diagnosis or accurate diagnosis with testing completed elsewhere. To classify these patients would require collection and review of medical records from outside institutions with patient permission, which is beyond the scope of this chart-based study, but a future research opportunity.

Conclusion

The current investigation determined that the positive predictive value for ICD coding of IIH to be 63% among patients with necessary testing performed. The findings are similar to the previously projected estimates for IIH ICD code accuracy. This low accuracy supports the concern for misclassification bias in studies identifying IIH cases based on ICD coding alone. This needs to be considered in the interpretation of this studies. We offer strategies for improving accuracy of case identification. Further research is needed to validate these strategies in other populations and to determine the root of coding error.