How complete are E-codes in commercial plan claims databases?

To the Editor

We advise caution in applying the claim-based algorithm developed by Callahan et al¹ to identify suicidal behavior using claims data. The Callahan et al. method uses external cause of injury codes (E-codes) in combination with diagnosis codes for poisoning derived from the International Classification of Diseases, 9th revision, Clinical Modification (ICD-9-CM) coding scheme to identify hospitalizations for suicide attempts.

In recent years there has been considerable concern that suicidal behavior is a potential adverse outcome of prescription drug use such as antidepressant and anticonvulsant agents.² Non-fatal, deliberate self-harms resulting in emergency department treatments and hospitalizations can be identified in administrative databases using E-codes.^3-5 These codes are part of the ICD-9-CM and are used to provide information about the cause and intent of an injury or poisoning. E-coding is mandatory in about half of US states,⁶ and the completeness of E-codes in state hospital discharge databases typically exceeds 90%.⁷ As part of a study of effects of safety warnings for antidepressants and suicidality in youth, we assessed the completeness of E-codes in commercial health plan databases.

Methods

Our analysis included 10 geographically distinct healthcare organizations in the Mental Health Research Network (MHRN)⁸ within the HMO Research Network (HMORN). The health plans had a combined population of 9 million enrollees in 2010. This analysis was part of a longitudinal study of effects of Food and Drug Administration warnings for antidepressants and suicidality in youth that was approved by the institutional review board of each participating organization.

We used data from the HMORN Virtual Data Warehouse (VDW).^9-13 The VDW is a common data model in which the data elements at each site are harmonized to enable collaborative, multi-site research projects. The VDW contains demographics, health plan enrollment, inpatient and outpatient utilization (e.g., diagnoses, procedures, service dates), and pharmacy dispensings. The quality of these data has been previously validated;^9-13 however, the reliability of E-codes in the VDW has not been assessed.

We calculated the completeness of E-codes, defined as the proportion of encounters with an injury/poisoning ICD-9-CM code that had a valid E-code indicating the cause for the encounter.³ As in prior research,³ we identified hospitalizations and ED visits with a primary or secondary diagnosis of injury/poisoning. We focused on injuries that are likely methods of deliberate self-harm: open wound injuries, superficial injuries, and poisonings. Because E-code collection and reporting requirements vary temporally and by region, we assessed E-code completeness rates from 2000 through 2010 by MHRN site and care setting. A V-code (supplemental information about factors influencing health service use) was introduced in 2005 indicating suicidal ideation (ICD-9: V62.84). In a sensitivity analysis, we calculated E-code completeness, while also including V62.84 that could be used in place of E-codes to identify a suicidal related encounter.

Results

Figure 1 presents E-code completeness rates in emergency department and hospital settings over time. E-code completeness varied widely across study sites (e.g., ranging from 7% to 92% in the emergency department setting in 2010), across treatment settings (e.g., ranging from 7% to 56% at one study site in 2010), and across years (e.g., ranging from 36% in 2000 to 92% in 2010 at one study site). Only two sites had consistent, reasonable levels of E-code recording over this period (range: 65% to 82%). Our investigation indicates the suicidal ideation code did not substitute or compensate for lack of E-codes with injury/poisoning diagnoses.

Figure 1.

Proportion of injury and poisoning encounters that had a valid E-code in (A) emergency rooms and (B) hospitals by study site (2000-2010)

Comment

In our analysis of VDW data from 10 HMORN sites between 2000 and 2010, we found that E-code completeness varied across study sites, across treatment settings, and across years of observation. There are several possible reasons for the low rates of and/or variability in E-code completeness observed: E-codes have no relevance for payments;³ not all diagnosis codes are transformed into the VDW from source data; and recording practice for E-codes may vary across sites, possibly due to state regulations, health plan policies, or the clinical software used. The incompleteness we observed in this study limits the usefulness of the available E-coded data. Other studies also found high missingness of E-codes in hospital and emergency department settings.^3,14-17

Despite the high positive predictive value of 85% reported by Callahan et al¹, there are two issues relating to the use of the algorithm: (1) the completeness of E-codes in the dataset, and (2) the dependence on valid E-coded data.

We agree with Callahan et al that it is important to develop and use alternative diagnosis codes that can identify suicide attempts. In the absence of complete E-codes, Patrick et al developed and tested algorithms for identifying hospitalizations for deliberate self-harm in a population aged 10 and over.³ This study used the US National Inpatient Sample data and data from British Columbia; both data sources had E-code completeness rates above 85 percent. The gold standard for deliberate self-harm was defined as hospitalizations with a diagnosis of E950-958. Patrick et al found that an algorithm combining diagnoses for psychiatric disorders (including depression) and injury/poisoning can produce a positive predictive value as high as 87.8% for identifying hospitalizations for deliberate self-harm (with specificity of 99.4% and sensitivity of 57.3%).

In the context of our study on the impact of Food and Drug Administration warnings for antidepressants and suicidality in youth, Patrick’s algorithm would introduce selection bias because rates of depression diagnosis declined after the warning.^24,25,28 A diagnosis of ‘poisoning by psychotropic agents’ alone outperformed other injury/poisoning types; its positive predictive value was 79.7% with specificity of 99.3% and sensitivity of 38.3%.³ While psychotropic drug poisoning underestimate rates of suicide attempts because of its low sensitivity, it can be useful for detecting suicide attempts in study settings that have low or inconsistent E-code rates over time. Such consistency is required for longitudinal analyses of trends in deliberate self-harms.

In summary, our analysis confirmed that E-codes were substantially incomplete in commercial insurance claims databases. We observed that E-code completeness varied widely across MHRN sites, across treatment settings, and over time. Completeness improved at some sites and deteriorated at other sites. Psychotropic drug poisonings may be useful for identifying deliberate self-harm requiring hospitalization in commercial plan databases when E-codes are missing. Further research is warranted to validate the reliability of algorithms using data from emergency departments. Importantly, greater efforts are needed to improve the use of external cause of injury codes in commercial health plan databases.