Validating Coding for the Identification of Pediatric Treatment Resistant Epilepsy Patients

Abstract

The International Classification of Diseases (ICD) system includes sub codes to indicate that an individual with epilepsy is treatment resistant. These codes would be a valuable tool to identify individuals for quality improvement and population health, as well as for recruitment into clinical trials. However, the accuracy of these codes is unclear. We performed a single center cross sectional study to understand the accuracy of ICD codes for treatment resistant epilepsy. We identified 344 individuals, roughly half with treatment resistant epilepsy The ICD code had a sensitivity of 90% (147 of 164) and specificity of 86% (155 of 180). The miscoding of children with refractory epilepsy was attributed to the following reasons: 5 patients had epilepsy surgery, 4 had absence epilepsy, 4 patients were seen by different providers, and 1 patient was most recently seen in movement disorders clinic. ICD codes accurately identify children with treatment resistant epilepsy.

Introduction

The International Classification of Diseases, Tenth Revision (ICD-10) allows for more accurate coding of epilepsy diagnoses. This has many benefits from the perspective of quality measure development, quality improvement, population health, and reimbursement. Utilizing the International Classification of Diseases (ICD) system to identify a population of epilepsy patients has been previously performed.¹ Limitations using this data and the need for validation have been well documented.² A study validating ICD coding for patients with infantile spasms was recently performed.³ However, no similar studies to validate the use of ICD coding for intractable or treatment resistant pediatric epilepsy patients have been performed. Validating the ICD codes used for intractable pediatric epilepsy patients will allow for easier estimates of population size, expedited referral to comprehensive epilepsy centers, and accurate reimbursement.

Treatment resistant epilepsy (intractable epilepsy) has been defined as failure of 2 appropriately chosen and properly titrated antiseizure medication (ASM) therapies.⁴ The implications of this definition are significant with a limited chance of seizure freedom occurring.⁵ To assist in improving patient access to other treatments, the American Academy of Neurology (AAN) developed a quality measure to improve referral of treatment resistant epilepsy patients to a comprehensive epilepsy center for further evaluation and management of all aspects of epilepsy care.⁶ Refinement of this measure methodology has since occurred.⁷ Challenges are present due to the difficulty in identifying these patients easily. A limitation of a 2015 study examining the utilization of pediatric epilepsy surgery revealed that the accuracy of ICD-9 codes in treatment resistant epilepsy had never been validated.⁸ This highlights the importance of establishing the accuracy of these codes for future research, as well as the identification of patients for possible clinical trials. Therefore, we performed an analysis to determine the usefulness of ICD coding for properly identifying patients with treatment resistant epilepsy.

Methods

Study Design

After obtaining Institutional Review Board approval from Nationwide Children's Hospital (NCH), we performed a retrospective cross-sectional chart review for patients to determine the accuracy of the use of an ICD code compared to a chart review of clinical information.

Setting

NCH is a freestanding pediatric hospital with 427 licensed beds, 2 freestanding emergency departments, and 7 urgent care facilities. NCH is the sole tertiary pediatric care facility in Central Ohio. Approximately, 3600 unique patients receive care for their epilepsy at NCH.

Cohort

A convenience sample of children in our epilepsy practice who had enrolled in prior observational studies, and who had at least 2 outpatient clinic visits, were used for this study. Data for chart review was extracted from the patients’ electronic health records (EHRs). Data was collected and stored in a password-protected spreadsheet. In an attempt for adequate clinical documentation, we verified that patients were seen at least 2 times in the outpatient neurology or epilepsy clinic within time span of 2015 to 2016. Three categories were examined to determine treatment resistance: history of ASM usage, past provider documentation within clinical notes, and a prior history of epilepsy surgery.

Chart Abstraction

ASMs were documented from the physician's note and medication record by 3 abstractors. In review of clinic visit documentation, the chart abstractor searched using a search tool for the terminology indicative of treatment resistance (intractable, treatment resistant, or refractory). If adequate documentation of patient intractability was absent, but the patient was previously on or had failed 3 ASMs, the patient was considered to be treatment resistant. The logic was based on the methodology used by the AAN quality measure.⁷

If the patient had surgery related to his/her epilepsy, including but not exclusive to corpus callosotomy and neurostimulation device placement, the patient was marked as treatment resistant unless documentation of seizure freedom was present in the medical record for 12 or more months.

If conflicting documentation with both “intractable” and “non-intractable” was in the clinical note, the latest note was used. If contradiction occurred in the same note with mention of intractability and nonintractability, then the number of medications or previous epilepsy surgery were considered to determine intractability.

The ICD code was pulled for the visit to determine the billing associated with the encounter and compared to the chart review information to determine correlation of the documentation. Statistical analysis was performed, using R, to determine sensitivity and specificity. A secondary chart review was performed by a board eligible child neurologist and neurophysiology fellow for patients where a misclassification occurred to gather more information as to why the mismatch occurred.

Results

The charts of 352 children ages 0 to 21 years with an ICD-9 or ICD-10 code of epilepsy (345.x or G40.x) with at least 2 outpatient clinical visits at Nationwide Children's Hospital were manually reviewed. Of these random sampling of charts, 164 (47%) were treatment resistant and 180 (51%) were not treatment resistant according to the ICD code. For 8 (2%) of the children, information was missing or incomplete for proper classification and validation of the information after chart review. Information in the record for us to be able to determine if they were treatment resistant was missing. For example, data was missing on the number of medications previously taken or lack of seizure frequency documentation to determine if the patient was seizure free at the time of the visit. These patients were grouped as unclassified.

Among the remaining 344 children, 172 had an ICD-9 or ICD-10 code for treatment resistance and 172 did not. (That these numbers were the same was coincidental, and not by design). When compared to chart review, the overall accuracy of the ICD code was 88% (302 correct of 344). The ICD code correctly identified 147 of 164 children with treatment resistant epilepsy (sensitivity of 90%). The absence of an intractable code correctly indicated absence of treatment resistance in 155 of 180 children (specificity of 86%).

Details about the patients who were misclassified is as shown in Table 1. Four children had resective epilepsy surgery. They were in the postoperative period and not billed as intractable as they were seizure free at time of the clinical visit and had been able to wean off of some of their ASMs. One patient had a vagus nerve stimulator (VNS) implanted for intractable epilepsy (based on chart review of medications) but was not billed as intractable. Four patients had a diagnosis of refractory absence epilepsy based on review of the clinical notes; however, these patients were not coded specifically as refractory absence. Five additional patients were treatment resistant based on chart review but were not coded this way. Four of the 5 that were coded incorrectly were patients not seen consistently by the same neurology provider and led to conflicting documentation. One of these 5 patients obtained care in a movement disorder clinic with the ICD code for seizures (780.39) at that visit which was the last visit; therefore, this visit was used based on our methodology described above. Upon deeper chart review, the patient was coded correctly using the intractable ICD code when seen previously by an epilepsy provider.

Table 1.

Treatment Resistant Misclassified as Not Treatment Resistant Analysis.

Category	Number of patients	Age	Epilepsy type	Current number of antiseizure medications
Resective surgery	4	6	Focal	1
		2	Focal	1
		4	Focal	1
		15	Focal	1
Vagus nerve stimulator (VNS)	1	14	Unknown	1
Refractory absence	4	12	Generalized	2
		11	Generalized	2
		11	Generalized	2
		9	Generalized	1
Incorrect code used	5	20	Generalized	3
		12	Focal	4
		9	Unknown	1
		17	Focal	2
		7	Generalized	3

Fourteen patients were identified by the EHR as intractable. On manual chart review, these patients were likely treatment resistant based on the clinical documentation in the note, but the clinical record did not clearly document failure of 2 ASMs. These patients were classified as treatment resistant epilepsy as they were exposed to at least 3 ASMs. We included these patients in Table 2. These patients were generally older and some of the patients had been on ASM years ago as a neonate, which were restarted years later for seizure recurrence.

Table 2.

Not Treatment Resistant Misclassified as Treatment Resistant Analysis.

Category	Number of patients	Age	Epilepsy type	Current number of antiseizure medications
Misclassified by electronic health record (EHR)	14	6	Generalized	1
		3	Generalized	1
		19	Focal	2
		19	Generalized	2
		19	Generalized	2
		19	Unknown	1
		16	Unknown	1
		17	Focal	1
		13	Generalized	1
		8	Focal	1
		8	Unknown	2
		12	Focal	1
		7	Focal	2
		9	Generalized	1

Discussion

Summary of Findings

Overall, the use of ICD coding is an accurate method to identify children with treatment resistant epilepsy. There are several factors that have contributed to the 14 miscoded intractable patients. For the 4 patients that had undergone either resective epilepsy surgery or VNS placement, there was a history of treatment resistance with greater than or equal to 3 medication exposures. However, these patients were able to taper off of some medications which may explain why they were no longer coded as intractable. There is no clear explanation for why the 4 patients with absence epilepsy were not more accurately coded as intractable absence epilepsy. The other 5 miscoded patients were seen by multiple providers, such as the patient seen in movement disorders clinic, which may have contributed to the misclassification of their diagnosis.

Prior Literature

Jette et al² identified similar results for their work on validation of ICD codes for epilepsy in general showing a 99% sensitivity and 70% specificity. Similarly, Reid et al⁹ showed ICD-9 and ICD-10 codes were both sensitive (89%) and specific (92%) for adult patients with epilepsy. In patients with infantile spasms, the ICD code achieved higher sensitivity (97%) and specificity (100%) than our review. For this publication, claims data provided by an accountable care organization (partners for kids) was used to validate coding. Six different algorithms were utilized and validated against an infantile spasm registry. The use of ICD code alone performed the best.³ Similarly, Reid et al showed ICD-9 and ICD-10 codes were both sensitive (89%) and specific (92%) for adult patients with epilepsy. One study used diagnosis codes from a Medicaid multi-state database to identify patients with Lennox-Gastaut syndrome (LGS) but the database did not contain complete clinical information, limiting the authors’ ability to validate the data.¹⁰ In this study, the use of medications commonly prescribed and approved for LGS along with the use of the ICD-9 treatment resistant code of 345.x1¹¹ was the methodology to determine these patients had LGS.

Alternative Algorithms

An alternative strategy to identify individuals with treatment resistance is to find individuals with prescriptions for 3 or more ASMs, as suggested in the updated AAN epilepsy quality measure set.⁷

Implications for Quality Measurement

In order to implement a quality measure into practice, it is important that reliability of the methodology occurs to ensure the measure is accurately quantifying the intended population. The validation of ICD-9 and ICD-10 codes of intractable epilepsy in the pediatric population provided by this study confirms that ICD codes can be used as a reliable quality measure. Validation of measure methodology is important to ensure the proper numerator and denominator for the measure can be calculated correctly.

Quality Improvement Implications

These data demonstrate that the sensitivity and specificity of intractable codes could be improved upon. We suspect the source of error with postsurgical patients derives from the unclear definition of postop remission.¹² If a patient has a history treatment resistance but then is seizure free on fewer than 2 medications after surgery, is the patient still intractable?

Gaps continue to exist in getting patients referred to a comprehensive epilepsy center. Therefore, the AAN created a quality measure with an attempt to focus on the importance of what such epilepsy centers can offer patients with treatment resistant epilepsy.⁷ By being able to identify these patients easier, a referral may occur more easily made as EHRs could proactively find such patients and trigger a referral using clinical decision support. In addition, proper identification of treatment resistant epilepsy patients can be used by centers to gather more necessary resources based on their local population and numbers. Quality improvement methodology could be used to improve the baseline performance of centers to ensure high quality care and newer treatments are identified for these patients. Further efforts to have patients evaluated for epilepsy surgery can also improve if the population of treatment resistant patients is known.

Clinical & Research Implications

These findings highlight the utility and validity of identifying children with intractable epilepsy who may benefit from referral to a comprehensive epilepsy center. Comprehensive epilepsy centers offer many advances and resources that can address the consequences of treatment resistant or difficult to control epilepsy, such as emerging therapies, clinical trials, and surgical intervention¹³ Delays in life altering treatment such as resective epilepsy surgery are well documented¹⁴ and less delay in surgical evaluation can ultimately lead to improved patient outcomes.¹⁵

Implications for the Definition of “Treatment Resistant”

The definition of treatment resistant can be confusing and unclear. Different definitions have been proposed in the medical literature.¹⁶ The International League Against Epilepsy uses the definition of failure from seizure freedom after a patient has been treated with 2 or more ASMs that were appropriately selected and titrated.⁴ We used 3 or more ASMs as the definition in this study to reduce the chance of mislabeling patients who either had medications that were not appropriately selected or titrated.

Implications for Health Services Research

Analysis on cost burden and other important health care determinants can be performed when utilizing a claims or cost database as previously performed for other epilepsy populations.^17–19 Accurate cost quantification for this subset of epilepsy patients can be helpful to fully understand the impact both to the patient/caregivers and also the health care system. Further, proper identification of treatment resistant patients allows for understanding of long-term impact and outcomes such as hospitalizations, resource utilization, and death. Such understand may inform future research or work to address these important areas.

Limitations

There are several limitations to this study. This is an analysis of coding data from a single center, which limits the generalizability of these findings. Additionally, the clinical documentation was incomplete for some patients, lacking details of the medication history or current regimes resulting regimen in 14 patients with an unclear diagnosis and could suggest other patients’ medication histories were not accurately recorded.

Conclusion

This study shows that ICD codes can be accurately used to identify pediatric patients with treatment resistant epilepsy. Replication of this data from other centers is needed. Validated diagnostic codes in this population provide researchers with an important tool for quality improvement, epidemiology, and identification of patients for highly specialized therapies like epilepsy surgery.