Abstract
PURPOSE
In order to assess the safety of psychotropic medication use in children and adolescents, it is critical to be able to identify suicidal behaviors from medical claims data and distinguish them from other injuries. The purpose of this study was to develop an algorithm using administrative claims data to identify medically-treated suicidal behavior in a cohort of children and adolescents.
DESIGN/METHODS
The cohort included 80,183 youth (6–18 years) enrolled in Tennessee’s Medicaid program from 1995–2006 who were prescribed antidepressants. Potential episodes of suicidal behavior were identified using external cause-of-injury codes (E-codes) and ICD-9-CM codes corresponding to the potential mechanisms of or injuries resulting from suicidal behavior. For each identified episode, medical records were reviewed to determine if the injury was self-inflicted and if intent to die was explicitly stated or could be inferred.
RESULTS
Medical records were reviewed for 2676 episodes of potential self-harm identified through claims data. Among 1162 episodes that were classified as suicidal behavior, 1117 (96%) had a claim for suicide & self-inflicted injury, poisoning by drugs, or both. The positive predictive value (PPV) of code groups to predict suicidal behavior ranged from 0–88% and improved when there was a concomitant hospitalization but with the limitation of excluding some episodes of confirmed suicidal behavior.
CONCLUSIONS
Nearly all episodes of confirmed suicidal behavior in this cohort of youth included an ICD-9-CM code for suicide or poisoning by drugs. An algorithm combining these ICD-9-CM codes and hospital stay greatly improved the PPV for identifying medically-treated suicidal behavior.
Keywords: Suicidal behavior, Adolescence, Administrative claims
Background
Suicidal thoughts and behavior are common among youth in the United States. Approximately 14% of high school students in the US report having had suicidal ideation and 7% report attempting suicide at least once in the prior year.1,2 The potentially paradoxical relationship between the use of antidepressant medication and suicidal behavior in adolescents, leading to a “black box” warning by the Food and Drug Administration in 2004, has heightened interest in identifying deliberate self-injury in general and suicidal ideation and behavior in particular.3,4 Identifying suicidal behavior among adolescents is challenging because medically-treated injuries in the child and adolescent population also include non-suicidal self-injurious behavior and unintentional injuries.2
The use of large administrative medical claims databases is a potentially important, but underutilized mechanism for identifying medically-treated suicidal behavior among children and adolescents, particularly because claims data allow for the study of relatively rare events among large populations and are not subject to recall bias.5 Little is known, though, about the predictive value for administrative claims to identify suicidal behavior and there are no studies focusing on adolescents, an age-group at high risk for suicidal behavior.6
The objective of this study was to determine the positive predictive value of administrative claims data to identify medically-treated suicidal behavior in a large cohort of children and adolescents treated with antidepressant medication and to develop an algorithm combining claims data and hospital length of stay to identify medically-treated suicidal behavior in this cohort.
Methods
This study was part of an ongoing, retrospective cohort study of antidepressant medication and medically-treated suicidal behavior in children and adolescents. The cohort included 80,183 children and adolescents, age 6–18 years, enrolled in Tennessee’s Medicaid program (TennCare) between 1995 and 2006 who: 1) filled new prescriptions for antidepressant medications (selective serotonin reuptake inhibitors, serotonin-norepinephrine reuptake inhibitors, atypical and cyclic antidepressants and monoamine oxidase inhibitors); 2) who had not filled prescriptions for an antidepressant in the prior 365 days; and 3) who did not have evidence of bipolar disorder, psychosis, or schizophrenia. Cohort members were followed until the earliest of the following occurrences: the last study day, the 19th birthday, death, loss of enrollment for 7 or more days, no filling of antidepressant prescription for 180 consecutive days or having a confirmed suicide attempt. Children who left the cohort could re-enter if they met eligibility requirements again. Potential episodes of suicidal behavior were identified using International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) codes and external cause-of-injury codes (E-codes), assigned by the treating facility, corresponding to the potential mechanisms of/or injuries resulting from suicidal behavior. We utilized codes (listed in Tables 1 and 2) that have been used in prior population-based studies of suicide epidemiology and also included a wider array of injury claims in order to assure more complete case ascertainment.7–9
Table 1.
Positive predictive value (PPV) of E-codes 950–959 for confirmed suicidal behavior among 6–18 year olds prescribed antidepressants
| ICD-9 Code | Description | Potential Cases Identified | Confirmed Suicidal Behavior | PPV (%) |
|---|---|---|---|---|
| E950–E959 | Suicide & self-inflicted injury | 718 | 593 | 82.6 |
| E950 | Suicide & self-inflicted Injury by poisoning by solid/liquid | 499 | 438 | 87.8 |
| E951-2 | Suicide & self-inflicted Injury by poisoning by gases and vapors | 8 | 5 | 62.5 |
| E953 | Suicide & self-inflicted Injury by hanging or strangulation | 21 | 20 | 95.0 |
| E955 | Suicide & self-inflicted Injury by firearms | 4 | 3 | 75.0 |
| E956 | Suicide & self-inflicted Injury by cutting and piercing | 158 | 119 | 75.3 |
| E957 | Suicide & self-inflicted Injury by jumping from high places | 3 | 2 | 66.7 |
| E958 | Suicide & self-inflicted Injury by other and unspecified means | 44 | 24 | 54.5 |
| E959 | Late effects of suicide & self-inflicted injury | 0 | -- | -- |
Table 2.
Positive predictive value (PPV) of ICD-9-CM codes for confirmed suicidal behavior among 6–18 year olds prescribed antidepressants
| ICD-9 Code | Description | Injury episodes identified | Confirmed Suicidal Behavior | PPV (%) |
|---|---|---|---|---|
| 994.7 | Asphyxiation & strangulation | 10 | 8 | 80.0 |
| 960-977 | Poisoning by drugs, medicinal & biological substances | 1559 | 951 | 61.0 |
| E980-E989 | Injury undetermined whether accidentally or purposely inflicted | 401 | 138 | 34.4 |
| E850-E858 | Accidental poisoning by drugs, medicinal substances, & biologicals | 221 | 69 | 31.2 |
| 879 | Open wound of other & unspecified sites, except limbs | 137 | 23 | 16.8 |
| 875 | Open wound of chest (wall) | 34 | 5 | 14.7 |
| 851 | Cerebral laceration & concussion | 17 | 2 | 11.8 |
| E860-E869 | Accidental poisoning by other solid & liquid substances, gases, & vapors | 75 | 6 | 8.0 |
| E922 | Accident caused by firearm & air gun missile | 43 | 1 | 2.3 |
| 860-869 | Internal injury of thorax, abdomen, & pelvis | 104 | 2 | 1.9 |
| E884.9 | Other fall from one level to another | 259 | 0 | 0 |
| 876 | Open wound of back | 36 | 0 | 0 |
| E815 | Other motor vehicle traffic accident involving collision on the highway | 32 | 0 | 0 |
| E882 | Fall from or out of building or other structure | 21 | 0 | 0 |
| 852 | Subarachnoid, subdural, & extra-dural hemorrhage, following injury | 15 | 0 | 0 |
| 348.4 | Compression of brain | 7 | 0 | 0 |
| 853 | Other & unspecified intracranial hemorrhage, following injury | 7 | 0 | 0 |
| E913 | Accidental mechanical suffocation | 1 | 0 | 0 |
No injury episodes were identified for the following claims: motor vehicle traffic accident with train (E810), due to loss of control without collision (E816), or for falls from ladders (E881), or cliff (E884.1).
The outcome of interest was confirmed suicidal behavior. The procedure for confirming suicidal behavior is shown in Figure 1. Medical records for each episode of potential suicidal behavior identified using the selected ICD-9-CM codes were requested from the facility where the subject received care, including records from emergency medical services, emergency departments, medical and psychiatric hospitalizations and, in episodes of death, autopsy reports. Trained nurses went to medical facilities throughout the state and obtained key sections of the medical records, removing all personal identifiers. Then, two investigators (STC, WOC) independently adjudicated the medical records using definitions and training examples from the Columbia Classification Algorithm of Suicide Assessment (C-CASA).10 The C-CASA, commissioned and utilized by the FDA in its assessments of SSRI’s, consists of eight categories to characterize suicidal, non-suicidal, and indeterminate events based on the presence of planned or realized self-injurious behavior and an assessment of stated or inferred intent to die from the injury.10 Suicidal episodes included completed and attempted suicide, preparatory action toward imminent suicidal behavior (e.g., interrupted and aborted attempts) and suicidal ideation. Two psychiatrists (DCF, RCS) reviewed a random sample of episodes as well as all cases for which there was disagreement in adjudication (3%) or episodes that were considered equivocal (8%).
Figure 1.
Process for identifying and confirming episodes of suicidal behavior
Less than 1% (22 of 3107) of the identified episodes were repeat events; therefore we present positive predictive values (PPV) defined as the number of confirmed suicidal events divided by the total number of episodes identified using specified administrative claims. Because the two most commonly used claim groupings, suicide & self-inflicted injury (E950–959) and poisoning by drugs, medicinal & biological substances (960–977), may be appropriately used for suicidal and non-suicidal injury, we also examined the utility of using hospital admission and length of stay data to improve the PPV of these claims. We combined these claims in hierarchical fashion with hospital length of stay data to develop an algorithm to identify medically-treated suicidal behavior.
Permission for use of the study data was granted by the Tennessee Department of Health and the TennCare Bureau. The study was approved by the Vanderbilt University Institutional Review Board.
Results
For the study period, 3107 episodes of potential suicidal behavior were identified from medical claims data. Of these, 2676 had medical records available for adjudication. Episodes without medical records included those with missing or incomplete documentation (11%) or those in which the hospital refused to release records (3%). Sociodemographic and administrative claims characteristics for these episodes were similar to those for episodes for which medical records were available. After review of the medical records, 1162 episodes were adjudicated as suicidal events.
The PPV of the E-codes for suicide & self-inflicted injury (E950–959) are shown in Table 1. Collectively, these E-codes had the highest overall PPV for confirmed suicidal behavior at 83%. Within this group of E-codes, the largest number of claims and the highest PPV (88%) were for claims for suicide & self-inflicted injury by poisoning (E950). The second most frequently identified claim in this group, suicide & self-inflicted injury by cutting and piercing (E956), had lower PPV for confirmed suicidal behavior of 75%.
The PPV’s for confirmed suicidal behavior of the other ICD-9-CM and E-codes utilized in this study were lower than those for claims for suicide & self-inflicted injury and were particularly poor for codes for falls or injuries and wounds to the thorax, abdomen, back and extremities (Table 2). Overall, there were 1559 injury episodes identified by a claim of poisoning by drugs medicinal and biologic substances (960–977) of which 951 were confirmed as suicidal behavior (PPV 61%).
The algorithm for identifying suicidal behavior derived from medical claims combined with hospital stay data is shown in Figure 2. The algorithm utilized a hierarchy of two claim groupings, suicide & self-inflicted injury (E950–959) and poisoning by drugs, medicinal & biological substances (960–977), one or both of which were present for 1117 of 1162 (96%) of the confirmed episodes of suicidal behavior in this cohort. Within the algorithm, the presence of any E-code for suicide & self-inflicted injury other than that for suicide & self-inflicted injury by cutting or piercing (E956) identified 560 episodes of which 474 were confirmed as suicidal behavior (PPV 85%). When a claim for suicide & self-inflicted injury by cutting or piercing (E956) also included a claim for a hospital stay of 1 or more days, the PPV for confirmed suicidal behavior increased from 75% to 90%. The PPV of an E-code for suicide & self-inflicted injury by cutting or piercing without a hospital stay was 52%. Within the algorithm, the PPV for episodes that did not include an E-code for suicide & self-inflicted injury was 29%, but included 569 episodes of confirmed suicidal behavior.
Figure 2.
Algorithm for identifying suicidal behavior from administrative claims data
In the absence of an E-code for suicide, the overall PPV for confirmed suicidal behavior of an ICD-9-CM code for poisoning (960–977) was 49%. When an ICD-9-CM claim for poisoning co-occurred with a claim for a hospital stay of 1 or more days, 526 episodes were identified of which 366 were confirmed as suicidal behavior (PPV 70%; data not shown). However, when a claim for poisoning co-occurred with a claim for a hospital stay of 2 or more days, the PPV was 83% (263 confirmed suicidal episodes/316 total episodes identified). Among episodes of poisoning without a 2 day hospitalization, the PPV for suicidal behavior was only 34%. However, 261 episodes of confirmed suicidal behavior had these claim characteristics.
There were 884 identified episodes that did not include a claim for E-code for suicide & self-inflicted injury or ICD-9-CM code for poisoning. Of these, there were only 45 episodes of confirmed suicidal behavior (PPV 5%).
The final algorithm, combining E-codes for suicide and self-inflicted behavior or ICD-9-CM codes for poisoning with hospital length of stay data, identified 974 episodes, of which 825 were confirmed as suicidal behavior (PPV 85%). The PPV of episodes that were not identified by the algorithm was only 20%. Modifications to the algorithm to improve PPV were accompanied by substantial reductions in case ascertainment.
Discussion
This study aimed to assess the utility of ICD-9-CM codes and supplemental E-codes for identifying episodes of suicidal behavior among adolescents treated with antidepressant medication and to develop an algorithm for identifying suicidal behavior using administrative claims and hospital stay data, balancing PPV and case ascertainment. Nearly all episodes of confirmed suicidal behavior in this cohort included an E-code for suicide & self-inflicted injury or an ICD-9-CM code for poisoning. The final algorithm, combining ICD-9-CM codes with hospital length of stay data, had a PPV for identifying suicidal behavior of 85%. If validated in other populations, this approach is likely to have utility in studies seeking to identify suicidal behavior using administrative claims data.
It is encouraging that administrative claims and hospital stay data can be used to identify suicidal behavior with relatively high PPV. Clinicians face challenges in determining whether an injury to a child or adolescent is self-inflicted, and if so, whether the injury was accompanied by intent to die.2,11 Characteristics of administrative claims codes also present challenges to identifying suicidal behavior.6 For example, an E-code for suicide & self-inflicted injury can be correctly utilized for any self-inflicted injury. In addition to suicidal behavior, adolescent self-injury may also include accidental injury or self-inflicted injuries intended to affect an internal state (e.g., cutting to relieve anxiety) or to affect external circumstances.12,13 In this cohort, only 51% of the episodes of confirmed suicidal behavior included an E-code for suicide and self-inflicted injury. The majority of these episodes included claims for suicide & self-inflicted injury by poisoning (which had higher PPV of 88%) and by cutting and piercing (which had lower PPV of 75%, because these behaviors are not always accompanied by suicidal intent). With claims for cutting and piercing, hospital admission served as a marker of suicidal intent and/or injury severity, thus the inclusion of a hospital stay of at least 1 day improved the PPV of this claim to identify suicidal behavior.
Among the 49% of episodes that did not have an E-code for suicide, the large majority had an ICD-9-CM code for poisoning. The PPV for this ICD-9-CM code to predict suicidal behavior was low at 49%, reflecting the inclusion of episodes of accidental poisoning or intentional ingestions to affect a change in internal state or external circumstances.14 As with the E-codes for suicide & self-inflicted injury, administrative data regarding hospital admission and length of stay data was useful to improve the PPV to predict suicidal behavior. Poisonings, whether intentional or unintentional, are often accompanied by brief hospital stays to allow for detoxification or evaluation for end-organ injury;15 thus, the combination of an ICD-9-CM code for poisoning and a concomitant hospital stay of 1 day did not significantly improve the PPV to identify suicidal behavior. However, when the claim was accompanied by a hospital stay of 2 or more days, the PPV increased to 83%.
That 29% of confirmed episodes of suicidal behavior were not captured by the primary algorithm, suggests that chart review is an important step to determine suicidal intent and optimize case ascertainment. In our study, this was particularly true for episodes identified by claims for self-injury by cutting, poisonings and unspecified injuries. Medical records review—particularly multi-level review as was performed in this study—is a time- and labor-intensive process. However, such review has the additional benefit of allowing for collection of covariates that are likely to be important in epidemiologic studies.
Among this study’s limitations are that the study population included youth from a single state that were enrolled in Medicaid and therefore may not be generalizable to youth residing in other states or to those with other insurance coverage. For example, the population of youth with Medicaid in Tennessee includes a smaller proportion of Hispanic youth than the national average.16 Surveillance data suggest that relative to white non-Hispanic youth, Hispanic youth are more likely to report having attempted suicide and seeking medical treatment for suicidal behavior.1 Still, Tennessee has considerable urban and rural diversity and during the study period approximately 30% of Tennessee’s children were covered by Medicaid. Tennessee is also among approximately half of US states to mandate E-code use. Studies show that on average, states with this mandate have higher rates of E-code use than states without such mandates.6,17 The study population is drawn from a cohort of patients who were prescribed antidepressants which may limit the applicability of these findings to the adolescent population as a whole. How administrative data might identify suicide attempts in youth who are not prescribed antidepressant medication warrants further research. However, given the increasing use of antidepressant agents in children and adolescents, understanding the characteristics of suicidal behavior among antidepressant users is important. Finally, the stigma of suicide and suicidal behavior may lead to underreporting of these behaviors in the medical record and the medical codes utilized in administrative claims data. To minimize this potential bias, we intentionally utilized a wide array of injury claims and adjudicated episodes as being suicidal if intent to die was stated or could be inferred. Thus, even in the absence or denial of a stated intent to die, any self-injury that was perceived by the subject as possibly being lethal (even if it was not) or any highly lethal act that was clearly not an accident could be adjudicated as a suicidal event.
In conclusion, our findings suggest that administrative claims data can be used to identify suicidal behavior in large cohort studies of adolescents. Algorithms to identify suicidal behavior should incorporate claims and length of stay data to improve positive predictive value.
Key points.
Little is known about the predictive value for administrative claims to identify suicidal behavior among adolescents, a population at high risk for such behavior.
In this large cohort of adolescence treated with antidepressant medication, 95% of all confirmed suicidal behavior included an E-code for suicide & self-inflicted injury or an ICD-9-CM code for poisoning.
Incorporating hospital stay data with these codes improved the positive predictive value for identifying suicidal behavior.
Footnotes
Potential conflicts of interest: The authors report that they have no conflicts to disclose.
Presented in part at the 2012 annual meetings of the Society for Adolescent Health and Medicine, New Orleans, LA and the Pediatric Academic Societies, Boston, MA.
References
- 1.Eaton DK, Kann L, Kinchen S, et al. Youth risk behavior surveillance--United States, 2007. MMWR Surveill Summ. 2008 Jun 6;57(4):1–131. [PubMed] [Google Scholar]
- 2.Hawton K, Saunders KE, O’Connor RC. Self-harm and suicide in adolescents. Lancet. 2012 Jun 23;379(9834):2373–2382. doi: 10.1016/S0140-6736(12)60322-5. [DOI] [PubMed] [Google Scholar]
- 3.Lofthouse N, Yager-Schweller J. Nonsuicidal self-injury and suicide risk among adolescents. Current opinion in pediatrics. 2009 Oct;21(5):641–645. doi: 10.1097/MOP.0b013e3283306825. [DOI] [PubMed] [Google Scholar]
- 4.Gibbons RD, Brown CH, Hur K, Davis J, Mann JJ. Suicidal thoughts and behavior with antidepressant treatment: reanalysis of the randomized placebo-controlled studies of fluoxetine and venlafaxine. Arch Gen Psychiatry. 2012 Jun;69(6):580–587. doi: 10.1001/archgenpsychiatry.2011.2048. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Spicer RS, Miller TR. Suicide acts in 8 states: incidence and case fatality rates by demographics and method. Am J Public Health. 2000 Dec;90(12):1885–1891. doi: 10.2105/ajph.90.12.1885. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Walkup JT, Townsend L, Crystal S, Olfson M. A systematic review of validated methods for identifying suicide or suicidal ideation using administrative or claims data. Pharmacoepidemiology and drug safety. 2012 Jan;21 (Suppl 1):174–182. doi: 10.1002/pds.2335. [DOI] [PubMed] [Google Scholar]
- 7.Centers for Disease Control and Prevention (U.S.), Centers for Medicare & Medicaid Services (U.S.). . DHHS publication; no (PHS) 04-1260. 6. Washington, D.C: U.S. Dept. of Health and Human Services, Centers for Disease Control and Prevention [U.S. G.P.O., Supt. Of Docs., distributor]; 2004. ICD-9-CM international classification of diseases, ninth revision, clinical modification. http://purl.access.gpo.gov/GPO/LPS8749. [Google Scholar]
- 8.Olfson M, Gameroff MJ, Marcus SC, Greenberg T, Shaffer D. National trends in hospitalization of youth with intentional self-inflicted injuries. Am J Psychiatry. 2005 Jul;162(7):1328–1335. doi: 10.1176/appi.ajp.162.7.1328. [DOI] [PubMed] [Google Scholar]
- 9.Garfinkel BD, Froese A, Hood J. Suicide attempts in children and adolescents. Am J Psychiatry. 1982 Oct;139(10):1257–1261. doi: 10.1176/ajp.139.10.1257. [DOI] [PubMed] [Google Scholar]
- 10.Posner K, Oquendo MA, Gould M, Stanley B, Davies M. Columbia Classification Algorithm of Suicide Assessment (C-CASA): classification of suicidal events in the FDA’s pediatric suicidal risk analysis of antidepressants. Am J Psychiatry. 2007 Jul;164(7):1035–1043. doi: 10.1176/appi.ajp.164.7.1035. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Dolan MA, Fein JA Committee on Pediatric Emergency M. Pediatric and adolescent mental health emergencies in the emergency medical services system. Pediatrics. 2011 May;127(5):e1356–1366. doi: 10.1542/peds.2011-0522. [DOI] [PubMed] [Google Scholar]
- 12.Wilkinson P, Goodyer I. Non-suicidal self-injury. Eur Child Adolesc Psychiatry. 2011 Feb;20(2):103–108. doi: 10.1007/s00787-010-0156-y. [DOI] [PubMed] [Google Scholar]
- 13.Moran P, Coffey C, Romaniuk H, et al. The natural history of self-harm from adolescence to young adulthood: a population-based cohort study. Lancet. 2012 Jan 21;379(9812):236–243. doi: 10.1016/S0140-6736(11)61141-0. [DOI] [PubMed] [Google Scholar]
- 14.Lin YR, Wu TK, Liu TA, Chou CC, Wu HP. Poison exposure and outcome of children admitted to a pediatric emergency department. World J Pediatr. 2011 May;7(2):143–149. doi: 10.1007/s12519-011-0267-7. [DOI] [PubMed] [Google Scholar]
- 15.Hollander JE, McCracken G, Johnson S, Valentine SM, Shih RD. Emergency department observation of poisoned patients: how long is necessary? Acad Emerg Med. 1999 Sep;6(9):887–894. doi: 10.1111/j.1553-2712.1999.tb01235.x. [DOI] [PubMed] [Google Scholar]
- 16.Kaiser Family Foundation. United States: Distribution of the Nonelderly with Medicaid by Race/Ethnicity, states (2010–2011) 2011 Jan 18; http://www.statehealthfacts.org/profileind.jsp?ind=158&cat=3&rgn=1. 2013.
- 17.Annest JL, Fingerhut LA, Gallagher SS, et al. Strategies to improve external cause-of-injury coding in state-based hospital discharge and emergency department data systems: recommendations of the CDC Workgroup for Improvement of External Cause-of-Injury Coding. MMWR. Recommendations and reports : Morbidity and mortality weekly report. Recommendations and reports / Centers for Disease Control. 2008 Mar 28;57(RR-1):1–15. [PubMed] [Google Scholar]